Keywords: Constrained Motion Planning , Task and Motion Planning , Task Planning
Abstract: Rearrangement puzzles are variations of rearrangement problems in which

the elements of a problem are potentially logically linked together. To

efficiently solve such puzzles, we develop a motion planning approach based on a new state space that is logically factored, integrating the capabilities of the robot through factors of simultaneously manipulatable joints of an object. Based on this factored state space, we propose less-actions RRT (LA-RRT), a planner which optimizes for a low number of actions to solve a puzzle. At the core of our approach lies a new path defragmentation method, which rearranges and optimizes consecutive edges to minimize action cost. We solve six rearrangement scenarios with a Fetch robot, involving planar table puzzles and an escape room scenario. LA-RRT significantly outperforms the next best asymptotically-optimal planner by 4.01 to 6.58 times improvement in final action cost.

Keywords: Motion and Path Planning
Abstract: Sample-based motion planning approaches, such as RRT*, have been widely adopted in robotics due to their support for high-dimensional state spaces and guarantees of completeness and optimality. This paper introduces an RRT* approach (ORRT*) that leverages opportunism to (1) find solutions quickly, (2) reduce wasted compute, and (3) improve data efficiency. The key insight of the approach is to make the most of compute when expanding the search tree by adding the last viable configurations found when connecting new nodes rather than rejecting the sampled nodes outright, allowing for more productive exploration of the space. We evaluate the proposed approach in a set of mobility and manipulator postural control domains, contrasting the performance of the opportunistic approach with state-of-the-art RRT* variants. Our analysis shows that such an approach has desirable characteristics and warrants further exploration.

Keywords: Constrained Motion Planning , Assembly , Path Planning for Multiple Mobile Robots or Agents
Abstract: Magnetic modular cubes are cube-shaped bodies with embedded permanent magnets. The cubes are uniformly controlled by a global time-varying magnetic field.

A 2D physics simulator is used to simulate global control and the resulting continuous movement of magnetic modular cube structures. We develop local plans , closed-loop control algorithms for planning the connection of two structures at desired faces. The global planner generates a building instruction graph for a target structure that we traverse in a depth-first-search approach by repeatedly applying local plans.

We analyze how structure size and shape affect planning time. The planner solves 80% of the randomly created instances with up to 12 cubes in an average time of about 200 seconds.

Keywords: Constrained Motion Planning , Redundant Robots , Motion and Path Planning
Abstract: Benefiting from its hyper-redundant structure, the biomimetic snake-like manipulator retains its remarkable flexibility even within confined spaces. However, its motion planning and control pose significant challenges. This paper imitates the winding uncoiling behavior of snakes to achieve controllable constrained path following. Firstly, based on control points, a recursive computational model and an equivalent planning angle model are established, enabling efficient and analytical determination of joint positions, collision regions, and motion parameters during the path following. Subsequently, the sliding control point algorithm and motion smoothing restriction algorithm are designed. The former ensures that the remaining segments during following strictly remain within the collision-free regions defined by the base and path controls, while the latter smooths the control parameters based on velocity and acceleration limitations. Finally, simulation and practical experiments demonstrate the feasibility of the proposed methods. The prototype that applied our method can reach targets and accomplish tasks, further validating the applicability of the snake-like manipulator.

Keywords: Constrained Motion Planning , Autonomous Vehicle Navigation , Intelligent Transportation Systems
Abstract: Smooth and safe speed planning is imperative for the successful deployment of autonomous vehicles. This paper presents a mathematical formulation for the optimal speed planning of autonomous driving, which has been validated in high-fidelity simulations and real-road demonstrations with practical constraints. The algorithm explores the inter-traffic gaps in the time and space domain using a breadth-first search. For each gap, quadratic programming finds an optimal speed profile, synchronizing the time and space pair along with dynamic obstacles. Qualitative and quantitative analysis in Carla is reported to discuss the smoothness and robustness of the proposed algorithm. Finally, we present a road demonstration result for urban city driving.

Keywords: Constrained Motion Planning , Motion and Path Planning
Abstract: Many applications require a robot to accurately track reference end-effector trajectories. Certain trajectories may not be tracked as single, continuous paths due to the robot's kinematic constraints or obstacles elsewhere in the environment. In this situation, it becomes necessary to divide the trajectory into shorter segments. Each such division introduces a reconfiguration, in which the robot deviates from the reference trajectory, repositions itself in configuration space, and then resumes task execution. The occurrence of reconfigurations should be minimized because they increase time and energy usage. In this paper, we present IKLink, a method for finding joint motions to track reference end-effector trajectories while executing the minimum number of reconfigurations. Our graph-based method generates a diverse set of Inverse Kinematics (IK) solutions for every waypoint on the reference trajectory and utilizes a dynamic programming algorithm to find the optimal motion by linking the IK solutions. We demonstrate the effectiveness of IKLink through a simulation experiment and an illustrative demonstration using a physical robot.

Keywords: Motion and Path Planning , Autonomous Vehicle Navigation , Legged Robots
Abstract: A Control Barrier Function (CBF)-based motion planning algorithm is proposed. The algorithm explores an unknown environment to reach a target point, providing velocity commands to the robot controller module. CBFs, along with a circulation inequality are used to generate safe paths toward the goal while preventing collisions with obstacles. The proposed global navigation scheme is experimentally verified on a quadruped platform to demonstrate safe, collision-free exploration over long distances.

Keywords: Constrained Motion Planning , Deep Learning Methods
Abstract: Constrained Motion Planning (CMP) aims to find a collision-free path between the given start and goal configurations on the kinematic constraint manifolds. These problems appear in various scenarios ranging from object manipulation to legged-robot locomotion. However, the zero-volume nature of manifolds makes the CMP problem challenging, and the state-of-the-art methods still take several seconds to find a path and require a computationally expansive path dataset for imitation learning. Recently, physics-informed motion planning methods have emerged that directly solve the Eikonal equation through neural networks for motion planning and do not require expert demonstrations for learning. Inspired by these approaches, we propose the first physics-informed CMP framework that solves the Eikonal equation on the constraint manifolds and trains neural function for CMP without expert data. Our results show that the proposed approach efficiently solves various CMP problems in both simulation and real-world, including object manipulation under orientation constraints and door opening with a high-dimensional 6-DOF robot manipulator. In these complex settings, our method exhibits high success rates and finds paths in sub-seconds, which is many times faster than the state-of-the-art CMP methods.

Keywords: Constrained Motion Planning , Aerial Systems: Applications , Robot Safety
Abstract: This paper offers a practical method for certifiably safe operations of an unmanned aerial vehicle (UAV) with limited power and computation, useful for real-time operations where the UAV is exposed to significant disturbances in non- convex free space. We propose a motion planning method based on the Explicit Reference Governor (ERG) framework to ensure the safety of a flying quadrotor UAV. From a small set of experiment data and assumptions on modeling errors, a Lyapunov function is synthesized by which an ERG is constructed to modify the UAV set-points. The method can handle polyhedral obstacles and constraints imposed on the maximum thrust of the UAV and its maximum tilt. We demonstrate the approach with extensive simulations and experiments using a Crazyflie 2.1.

Keywords: Swarm Robotics , Aerial Systems: Mechanics and Control
Abstract: We present distributed distance-based control (DDC), a novel approach for controlling a multi-agent system, such that it achieves a desired formation, in a resource-constrained setting. Our controller is fully distributed and only requires local state-estimation and scalar measurements of inter-agent distances. It does not require an external localization system or inter-agent exchange of state information. Our approach uses spatial-predictive control (SPC), to optimize a cost function given strictly in terms of inter-agent distances and the distance to the target location. In DDC, each agent continuously learns and updates a very abstract model of the actual system, in the form of a dictionary of three independent key-value pairs (delta s, delta d), where delta d is the partial derivative of the distance measurements along a spatial direction delta s. This is sufficient for an agent to choose the best next action. We validate our approach by using DDC to control a collection of Crazyflie drones to achieve formation flight and reach a target while maintaining flock formation.

Keywords: Swarm Robotics , Aerial Systems: Perception and Autonomy , Multi-Robot Systems
Abstract: In this paper, we present a swarm robotics control and coordination approach that can be used for locating a moving target or source in a GNSS-denied indoor setting. The approach is completely on-board and can

be deployed on nano-drones such as the Crazyflies. The swarm acts on a simple set of rules to identify and trail a dynamically changing source

gradient. To validate the effectiveness of our approach, we conduct experiments to detect the maxima of the dynamic gradient, which was implemented with a set of lights turned on and off with a time-varying pattern. Additionally, we introduce also a minimalistic fully onboard obstacle avoidance method, and assess the flexibility of our method by introducing an obstacle into the environment. The strategies rely on local interactions among UAVs, and the sensing of the source happens only at the individual level and is scalar, making it a viable option for UAVs with limited capabilities. Our method is adaptable to other swarm platforms with only minor parameter adjustments. Our findings demonstrate the potential of this approach as a flexible solution to tackle such tasks in constrained GNSS-denied indoor environments successfully.

Keywords: Swarm Robotics , Autonomous Agents , Engineering for Robotic Systems
Abstract: Research with swarm robotics systems can be complicated, time-consuming, and often expensive in terms of space and resources. The situation is even worse for studies involving multiple, possibly heterogeneous robot swarms. Augmented reality can provide an interesting solution to these problems, as demonstrated by the ARK system (Augmented Reality for Kilobots), which enhanced the experimentation possibilities with Kilobots, also relieving researchers from demanding tracking and logging activities. However, ARK is limited in mostly enabling experimentation with a single swarm. In this paper, we introduce M-ARK, a system to support studies on multi-swarm interaction. M-ARK is based on the synchronisation over a network connection of multiple ARK systems, whether real or simulated, serving a twofold purpose: (i) to study the interaction of multiple, possibly heterogeneous swarms, and (ii) to enable a gradual transition from simulation to reality. Moreover, M-ARK

enables the interaction between swarms dislocated across multiple labs worldwide, encouraging scientific collaboration and advancement in multi-swarm interaction studies.

Keywords: Swarm Robotics , Cellular and Modular Robots , Biologically-Inspired Robots
Abstract: Various robot platforms have been developed for investigating new algorithms for swarm robotics. Morphogenetic engineering in robot swarms, however, proposes new requirements for platforms: precise motion control, physical interactions with the environment or neighbor robots, and functionalized shells. Few current platforms fulfill all the above characteristics. Here, we present Morphobot, a robot platform for morphogenetic engineering in swarm robotics. Its direct current coreless motors provide physical support and strong power, meanwhile, these needle-like motors also enhance the physical interactions among robots. Each Morphobot has a changeable shell. It is functionalized for programming local interactions, through physical contact or communication, among Morphobots. We characterized the mobility of Morphobot to test its capability of moving and physically interacting with its neighbors. To demonstrate its advantages in the morphogenesis of robot swarms, we designed two morphogenetic engineering experiments. The results revealed that swarms of Morphobots can form patterns via physical interactions and optical communications.

Keywords: Swarm Robotics , Cellular and Modular Robots , Mechanism Design
Abstract: FireAntV3 uses a refined version of the 3D Continuous Docks to attach to other such docks at any location at any orientation with simple control and without alignment. The robot improves upon previous FireAnt-series robots by redesigning the locomotion drive system to improve mechanical and attachment reliability while also reducing the number of motors from six to three. We also expand the sensory capabilities of FireAntV3 to enable the robot to sense forces, sense the direction to a light source, and to sense contacting neighbors using vibrations. We validate this robot through full-robot tests demonstrating phototaxis and neighbor-detecting behavior. This paper also describes the method for manufacturing the continuous docks in a variety of geometries.

Keywords: Swarm Robotics , Cellular and Modular Robots , Micro/Nano Robots
Abstract: Natural and robotic swarms often exhibit non-reciprocal interactions; agents do not exhibit equal and opposite forces on each other. By studying the effects of reciprocal and non-reciprocal interactions we are better able to design emergent behaviors in robot collectives composed of agents that exert attractive and repulsive forces on each other. Moreover, by controlling agent-specific coupling forces on-demand, we can enable a collective to exhibit desired behaviors previously not possible. We use a general form of the swarming oscillator, swarmalator, model to study reciprocal and non-reciprocal interactions among agents that affect each other's motions over long and short distances, we use non-reciprocal coupling to elicit collective locomotion toward or away from target sites, and we use the control barrier function method to optimize the non-reciprocal interactions for a desired spatial formation. This work addresses the interests of the active matter, swarm robotics, and control barrier functions communities and demonstrates various collective behaviors with strong potential to be realized in macro- and micro- length scale robot swarms.

Keywords: Swarm Robotics , Evolutionary Robotics
Abstract: Automatic design is a promising approach to realizing robot swarms. Given a mission to be performed by the swarm, an automatic method produces the required control software for the individual robots. Automatic design has concentrated on missions that a swarm can execute independently, interacting only with a static environment and without the involvement of other active entities. In this paper, we investigate the design of robot swarms that perform their mission by interacting with other robots that populate their environment. We frame our research within robot shepherding: the problem of using a small group of robots—the shepherds—to coordinate a relatively larger group—the sheep. In our study, the group of shepherds is the swarm that is automatically designed, and the sheep are pre-programmed robots that populate its environment. We use automatic modular design and neuroevolution to produce the control software for the swarm of shepherds to coordinate the sheep. We show that automatic design can leverage mission-specific interaction strategies to enable an effective coordination between the two groups.

Keywords: Swarm Robotics , Software-Hardware Integration for Robot Systems , Aerial Systems: Applications
Abstract: In this work we develop a software-in-the-loop simulator platform for Crazyflie nano quadrotor drone fleets. One of the challenges in maintaining a large fleet of drones is ensuring that the fleet performs its task as expected without collision, and this becomes more challenging as the number of drones scales, possibly into the hundreds. Software-in-the-loop simulation is an important component in verifying that drone fleets operate correctly and can significantly reduce development time. The simulator interface that we develop runs an instance of the Crazyflie flight stack firmware for each individual drone on a commercial, desktop machine along with a sensors and communication plugin on Gazebo Sim. The plugin transmits simulated sensor information to the firmware along with a socket link interface to run external scripts that would be run on a ground station during hardware deployment. The plugin simulates a radio communication delay between the drones and the ground station to test offboard control algorithms and high-level fleet commands. To validate the proposed simulator, we provide a case study of decentralized model predictive control (MPC) that is run on a ground station to command a fleet of sixteen drones to follow a specified trajectory. We first run the controller on the simulator interface to verify performance and robustness of the algorithm before deployment to a Crazyflie hardware experiment in the Georgia Tech Robotarium.

Keywords: Omnidirectional Vision
Abstract: Multi-view stereo omnidirectional distance estimation usually needs to build a cost volume with many hypothetical distance candidates. The cost volume building process is often computationally heavy considering the limited resources a mobile robot has. We propose a new geometry-informed way of distance candidates selection method which enables the use of a very small number of candidates and reduces the computational cost. We demonstrate the use of the geometry-informed candidates in a set of model variants. We find that by adjusting the candidates during robot deployment, our geometry-informed distance candidates also improve a pre-trained model's accuracy if the extrinsics or the number of cameras changes. Without any re-training or fine-tuning, our models outperform models trained with evenly distributed distance candidates. Models are also released as hardware-accelerated versions with a new dedicated large-scale dataset. The project page, code, and dataset can be found at https://theairlab.org/gicandidates/.

Keywords: Omnidirectional Vision
Abstract: On the one hand, cameras of conventional field-of-view usually considered in computer vision and robotics are very often modeled as a pinhole plus possibly a distortion model. On the other hand, there is a large variety of models for panoramic cameras. Many camera models have been proposed for fisheye cameras, catadioptric cameras, and super fisheye cameras. But in both cases, few models offer the possibility of converting them into another model.

This paper contributes to filling this gap in, to allow an algorithm designed with a projection model to accept data of a camera calibrated with another model. So, a pre-existing data set can be used without having to recalibrate the camera. We provide the methodology and mathematical developments for three conversions considering three different types of cameras that are evaluated with respect to calibration and within a visual Simultaneous Localization And Mapping benchmark. The source code of the camera model conversions studied in this paper is shared within the libPeR library for Perception in Robotics: https://github.com/PerceptionRobotique/libPeR_base.

Keywords: Range Sensing , Deep Learning for Visual Perception , Aerial Systems: Perception and Autonomy
Abstract: Abstract— In this work, we present RadCloud, a novel real time framework for directly obtaining higher-resolution lidar-like 2D point clouds from low-resolution radar frames on resource-constrained platforms commonly used in unmanned aerial and ground vehicles (UAVs and UGVs, respectively); such point clouds can then be used for accurate environmental mapping, navigating unknown environments, and other robotics tasks. While high-resolution sensing using radar data has been previously reported, existing methods cannot be used on most UAVs, which have limited computational power and energy; thus, existing demonstrations focus on offline radar processing. RadCloud overcomes these challenges by using a radar configuration with 1/4th of the range resolution and employing a deep learning model with 2.25× fewer parameters. Additionally, RadCloud utilizes a novel chirp-based approach that makes obtained point clouds resilient to rapid movements (e.g., aggressive turns or spins), that commonly occur during UAV flights. In real-world experiments, we demonstrate the accuracy and applicability of RadCloud on commercially available UAVs and UGVs, with off-the-shelf radar platforms on-board.

Keywords: Range Sensing , Localization , Mapping
Abstract: This paper is about 3D pose estimation on LiDAR scans with extremely minimal storage requirements to enable scalable mapping and localisation. We achieve this by clustering all points of segmented scans into semantic objects and representing them only with their respective centroid and semantic class. In this way, each LiDAR scan is reduced to a compact collection of four-number vectors. This abstracts away important structural information from the scenes, which is crucial for traditional registration approaches. To mitigate this, we introduce an object-matching network based on self- and cross-correlation that captures geometric and semantic relationships between entities. The respective matches allow us to recover the relative transformation between scans through weighted Singular Value Decomposition (SVD) and RANdom SAmple Consensus (RANSAC). We demonstrate that such representation is sufficient for metric localisation by registering point clouds taken under different viewpoints on the KITTI dataset, and at different periods of time localising between KITTI and KITTI-360. We achieve accurate metric estimates comparable with state-of-the-art methods with almost half the representation size, specifically 1.33 kB on average.

Keywords: Range Sensing , Localization , Mapping
Abstract: Limited by the working principles, LiDAR-SLAM systems suffer from the degeneration phenomenon in environments such as long corridors and tunnels, due to the lack of sufficient geometric features for frame-to-frame matching. The accuracy and sensitivity of existing degeneracy detection methods need to be further improved. In this paper, we propose a novel method for degeneracy detection using local geometric models based on point-to-distribution matching. To obtain an accurate description of local geometric models, an adaptive adjustment of voxel segmentation according to the point cloud distribution and density is designed. The codes of the proposed method is open-source and available at https://github.com/jisehua/Degenerate-Detection.git. Experiments with public datasets and self-build robots were conducted to evaluate the methods. The results exhibit that our proposed method achieves higher accuracy than the other existing approaches. Applying our proposed method is beneficial for improving the robustness of the LiDAR-SLAM systems.

Keywords: Range Sensing , Localization , Mapping
Abstract: Establishing the correspondences between newly acquired points and historically accumulated data (i.e., map) through nearest neighbors search is crucial in numerous robotic applications. However, static tree data structures are inadequate to handle large and dynamically growing maps in real-time. To address this issue, we present the i-Octree, a dynamic octree data structure that supports both fast nearest neighbor search and real-time dynamic updates, such as point insertion, deletion, and on-tree down-sampling. The i-Octree is built upon a leaf-based octree and has two key features: a local spatially continuous storing strategy that allows for fast access to points while minimizing memory usage, and local on-tree updates that significantly reduce computation time compared to existing static or dynamic tree structures. The experiments show that i-Octree outperforms contemporary state-of-the-art approaches by achieving, on average, a 19% reduction in runtime on real-world open datasets.

Keywords: Range Sensing , Mapping , Data Sets for SLAM
Abstract: Depth perception is considered an invaluable source of information in the context of 3D mapping and various robotics applications. However, point cloud maps acquired using consumer-level light detection and ranging sensors (lidars) still suffer from bias related to local surface properties such as measuring beam-to-surface incidence angle. This fact has recently motivated researchers to exploit traditional filters, as well as the deep learning paradigm, in order to suppress the aforementioned depth sensors error while preserving geometric and map consistency details. Despite the effort, depth correction of lidar measurements is still an open challenge mainly due to the lack of clean 3D data that could be used as ground truth. In this paper, we introduce two novel point cloud map consistency losses, which facilitate self-supervised learning on real data of lidar depth correction models. Specifically, the models exploit multiple point cloud measurements of the same scene from different view-points in order to learn to reduce the bias based on the constructed map consistency signal. Complementary to the removal of the bias from the measurements, we demonstrate that the depth correction models help to reduce localization drift. Additionally, we release a data set that contains point cloud scans captured in an indoor corridor environment with precise localization and ground truth mapping information.

Keywords: Range Sensing , Mapping , Field Robots
Abstract: The precise point cloud ground segmentation is a crucial prerequisite of virtually all perception tasks for LiDAR sensors in autonomous vehicles. Especially the clustering and extraction of objects from a point cloud usually relies on an accurate removal of ground points. The

correct estimation of the surrounding terrain is important for aspects of the drivability of a surface, path planning, and obstacle prediction. In this article, we propose our system GroundGrid which relies on 2D elevation maps to solve the terrain estimation and point cloud ground segmentation problems. We evaluate the ground segmentation

and terrain estimation performance of GroundGrid and compare it to other state-of-the-art methods using the SemanticKITTI dataset and a novel evaluation method relying on airborne LiDAR scanning. The results

show that GroundGrid is outperforming other state-of-the-art systems with an average IoU of 94.78% while maintaining a high run-time performance of 171Hz. The source code is available at https://github.com/dcmlr/groundgrid

Keywords: Range Sensing , RGB-D Perception , Mapping
Abstract: Reconstructing accurate object shapes based on single image inputs is still a critical and challenging task, mainly due to the potential shape ambiguity and occlusion. Most existing single image 3D reconstruction approaches, either trained on stereo setting or structure-from-motion, estimate 2.5D visible models which generally reconstruct one viewpoint of objects. We propose a method to leverage both the general Morphable Model on common objects and a multi-view synthesis-based shape-from-silhouette model to reconstruct complete object shapes. We use the proposed method to exploit strong geometric and perceptual cues in 3D shape reconstruction. During the inference, the trained model is able to produce high-quality and complete meshes with finely detailed structures from a 2D image captured from arbitrary perspectives. The proposed method is evaluated on both large-scale synthetic ShapeNet and real-world Pascal 3D+ and Pix3D datasets. The proposed work achieves state-of-the-art results compared with other recent self-supervised methods. Moreover, it shows a good capability of being applied in the unseen object reconstruction tasks.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Collision Avoidance , Deep Learning Methods
Abstract: To date, commercial industrial robots only provide multi-robot coordination for their own fleet of robots and treat robots from other vendors as general obstacles. The ability to enable robots from different vendors to co-exist in the same space is crucial to prevent vendor lock-in. We present the first decentralized system that achieves coordination between a heterogeneous fleet of black box robots for which the internals of the navigation stack are presumed unmodifiable. Our system, which we call CODAK, achieves the coordination by relying on minimum set of interfaces that are commonly available on most industrial and service robots. For each robot, CODAK uses a trained recurrent neural network to anticipate collisions from externally observable metrics. Anticipated collisions are avoided using a simple, but yet effective, concurrency control scheme. We run a series of experiments in simulation and with real robots to demonstrate CODAK’s ability to enable safe navigation in different environments. We also experimentally compare CODAK with previously published white-box solutions to evaluate the penalty of black-box constraint.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Cooperating Robots , Model Learning for Control
Abstract: Guided trajectory planning involves a leader robot strategically directing a follower robot to collaboratively reach a designated destination. However, this task becomes notably challenging when the leader lacks complete knowledge of the follower's decision-making model. There is a need for learning-based methods to effectively design the cooperative plan. To this end, we develop a Stackelberg game-theoretic approach based on the Koopman operator to address the challenge. We first formulate the guided trajectory planning problem through the lens of a dynamic Stackelberg game. We then leverage Koopman operator theory to acquire a learning-based linear system model that approximates the follower's feedback dynamics. Based on this learned model, the leader devises a collision-free trajectory to guide the follower using receding horizon planning. We use simulations to elaborate on the effectiveness of our approach in generating learning models that accurately predict the follower's multi-step behavior when compared to alternative learning techniques. Moreover, our approach successfully accomplishes the guidance task and notably reduces the leader's planning time to nearly half when contrasted with the model-based baseline method.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Cooperating Robots , Multi-Robot Systems
Abstract: In this article, we consider a multi-agent path planning problem in a partially impeded environment. The impeded environment is represented by a graph with select road segments(edges) in disrepair impeding vehicular movement in the road network. A primary vehicle, which we refer to as a convoy, wishes to travel from a starting location to a destination while minimizing some accumulated cost. The convoy may traverse an impeded edge for an additional cost (associated with repairing the edge) than if it were unimpeded. A support vehicle, which we refer to as a service vehicle, is simultaneously deployed to assist the convoy by repairing edges, reducing the cost for the convoy to traverse those edges. The convoy is permitted to wait at any vertex to allow the service vehicle to repair an edge. The service vehicle is permitted to terminate its path at any vertex. The goal is then to find a pair of paths so the convoy reaches its destination while minimizing the total time (cost) the two vehicles are active, including any time the convoy waits. We refer to this problem as the Assisted Shortest Path Problem (ASPP). We present a generalized permanent labeling algorithm (GPLA) to find an optimal solution for the ASPP. We also introduce additional modifications to the labeling algorithm to significantly improve the computation time and refer to the modified labeling algorithm as GPLA*. Computational results are presented to illustrate the effectiveness of GPLA* in solving the ASPP.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Distributed Robot Systems , Autonomous Agents
Abstract: Information-based coverage directs robots to move over an area to optimize a pre-defined objective function based on some measure of information. Our prior work determined that the spectral decomposition of an information map can be used to guide a set of heterogeneous agents, each with different sensor and motion models, to optimize coverage in a target region, based on a measure called ergodicity. In this paper, we build on this insight to construct a reinforcement learning formulation of the problem of allocating heterogeneous agents to different search regions in the frequency domain. We relate the spectral coefficients of the search map to each other in three different ways. The first method maps agents to pre-defined sets of spectral coefficients. In the second method, each agent learns a weight distribution over all spectral coefficients. Finally, in the third method, each agent learns weight distributions as parameterized curves over coefficients. Our numerical results demonstrate that distributing and assigning coverage responsibilities to agents depending on their sensing and motion models leads to 40%, 51%, and 46% improvement in coverage performance as measured by the ergodic metric, and 15%, 22%, and 20% improvement in time to find all targets in the search region, for the three methods respectively.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Human-Aware Motion Planning , Human Factors and Human-in-the-Loop
Abstract: In public spaces shared with humans, ensuring multi-robot systems navigate without collisions while respecting social norms is challenging, particularly with limited communication. Although current robot social navigation techniques leverage advances in reinforcement learning and deep learning, they frequently overlook robot dynamics in simulations, leading to a simulation-to-reality gap. In this paper, we bridge this gap by presenting a new multi-robot social navigation environment crafted using Dec-POSMDP and multi-agent reinforcement learning. Furthermore, we introduce SAMARL: a novel benchmark for cooperative multi-robot social navigation. SAMARL employs a unique spatial-temporal transformer combined with multi-agent reinforcement learning. This approach effectively captures the complex interactions between robots and humans, thus promoting cooperative tendencies in multi-robot systems. Our extensive experiments reveal that SAMARL outperforms existing baseline and ablation models in our designed environment. Demo videos for this work can be found at: https://sites.google.com/view/samarl

Keywords: Path Planning for Multiple Mobile Robots or Agents , Intelligent Transportation Systems , Motion and Path Planning
Abstract: Anticipating possible future deployment of connected and automated vehicles (CAVs), cooperative autonomous driving at intersections has been studied by many works in control theory and intelligent transportation across decades. Simultaneously, recent parallel works in robotics have devised efficient algorithms for multi-agent path finding (MAPF), though often in environments with simplified kinematics. In this work, we hybridize insights and algorithms from MAPF with the structure and heuristics of optimizing the crossing order of CAVs at signal-free intersections. We devise an optimal and complete algorithm, Order-based Search with Kinematics Arrival Time Scheduling (OBS-KATS), which significantly outperforms existing algorithms, fixed heuristics, and prioritized planning with KATS. The performance is maintained under different vehicle arrival rates, lane lengths, crossing speeds, and control horizon. Through ablations and dissections, we offer insight on the contributing factors to OBS-KATS's performance. Our work is directly applicable to many similarly scaled traffic and multi-robot scenarios with directed lanes.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Optimization and Optimal Control , Cooperating Robots
Abstract: This paper addresses the problem of the communication of optimally compressed information for mobile robot path-planning. In this context, mobile robots compress their current local maps to assist another robot in reaching a target in an unknown environment. We propose a framework that sequentially selects the optimal level of compression, guided by the robot's path, by balancing map resolution and communication cost. Our approach is tractable in close-to-real scenarios and does not necessitate prior environment knowledge. We design a novel decoder that leverages compressed information to estimate the unknown environment via convex optimization with linear constraints and an encoder that utilizes the decoder to select the optimal compression. Numerical simulations are conducted in a large close-to-real map and a maze map and compared with two alternative approaches. The results confirm the effectiveness of our framework in assisting the robot reach its target by reducing transmitted information, on average, by approximately 50%, while maintaining satisfactory performance.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Planning, Scheduling and Coordination
Abstract: In multi-drone systems such as drone light shows, drones move in formation while avoiding collisions. However, few existing formation planning algorithms consider the wind fields of drones during planning. Since the wind field effect is prominent when drones have to fly close to each other, we cannot ignore the effect during planning. In this paper, we extend the reservation system in autonomous intersection management for grid-based formation planning by including a new type of reservation called non-exclusive reservations specifically for handling wind fields. We train a deep learning model to predict the deviation of a drone's trajectory when the drone enters the wind field of another drone and then use the reservation grid to prevent collision. Based on the reservation system, we develop a new formation planning algorithm that focuses on adjusting the start times of motion plans to avoid collision. Our experimental results show that trajectory prediction can help make better decisions in task assignments for minimizing makespans.

Keywords: Path Planning for Multiple Mobile Robots or Agents , Probability and Statistical Methods , Environment Monitoring and Management
Abstract: This paper addresses multi-robot informative path planning (IPP) for environmental monitoring. The problem involves determining informative regions in the environment that should be visited by robots to gather the most information about the environment. We propose an efficient sparse Gaussian process-based approach that uses gradient descent to optimize paths in continuous environments. Our approach efficiently scales to both spatially and spatio-temporally correlated environments. Moreover, our approach can simultaneously optimize the informative paths while accounting for routing constraints, such as a distance budget and limits on the robot's velocity and acceleration. Our approach can be used for IPP with both discrete and continuous sensing robots, with point and non-point field-of-view sensing shapes, and for both single and multi-robot IPP. We demonstrate that the proposed approach is fast and accurate on real-world data.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , Visual Learning
Abstract: Low-light enhancement and deblurring is vital for high-level vision-related nighttime tasks. Most existing cascade and joint enhancement methods may provide undesirable results, suffering from severe artifacts, deteriorating blur, and unclear details. In this paper, we propose a novel ambiguity-aware network (ASP-LED) with structural priors, including high-frequency and edge, to enable effective image representation learning for joint low-light enhancement and deblurring. Specifically, we employ a Transformer backbone to explore the global clues of the image. To compensate for the inadequate local detail optimization, we propose a multi-patch perception pyramid block that models the correlation between different size patches and ambiguity, and identifies non-uniform deblurring spatial features, facilitating the reconstruction of potential high-frequency and edge information. Furthermore, a prior-guided reconstruction block based on the parallel attention mechanism is present to adaptively correct global image with statistical features, which helps guide the model to refine sharp texture and structure. Extensive experiments performed on simulated and real-world datasets demonstrate the efficacy of our proposed method in restoring low-light blurry images with increased visual perception compared to state-of-the-art methods.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , Visual Learning
Abstract: Towards flexible object-centric visual perception, we propose a one-shot instance-aware object keypoint (OKP) extraction approach, AnyOKP, which leverages the powerful representation ability of pretrained vision transformer (ViT), and can obtain keypoints on multiple object instances of arbitrary category after learning from a support image. An off-the-shelf petrained ViT is directly deployed for generalizable and transferable feature extraction, which is followed by training-free feature enhancement. The best-prototype pairs (BPPs) are searched for in support and query images based on appearance similarity, to yield instance-unaware candidate keypoints. Then, the entire graph with all candidate keypoints as vertices are divided into sub-graphs according to the feature distributions on the graph edges. Finally, each sub-graph represents an object instance. AnyOKP is evaluated on real object images collected with the cameras of a robot arm, a mobile robot, and a surgical robot, which not only demonstrates the cross-category flexibility and instance awareness, but also show remarkable robustness to domain shift and viewpoint change.

Keywords: Deep Learning for Visual Perception , Computer Vision for Transportation , Visual Learning
Abstract: 3D occupancy prediction holds significant promise in the fields of robot perception and autonomous driving, which quantifies 3D scenes into grid cells with semantic labels. Recent works mainly utilize complete occupancy labels in 3D voxel space for supervision. However, the expensive annotation process and sometimes ambiguous labels have severely constrained the usability and scalability of 3D occupancy models. To address this, we present RenderOcc, a novel paradigm for training 3D occupancy models only using 2D labels. Specifically, we extract a NeRF-style 3D volume representation from multi-view images, and employ volume rendering techniques to establish 2D renderings, thus enabling direct 3D supervision from 2D semantics and depth labels. Additionally, we introduce an Auxiliary Ray method to tackle the issue of sparse viewpoints in autonomous driving scenarios, which leverages sequential frames to construct comprehensive 2D rendering for each object. To our best knowledge, RenderOcc is the first attempt to train multi-view 3D occupancy models only using 2D labels, reducing the dependence on costly 3D occupancy annotations. Extensive experiments demonstrate that RenderOcc achieves comparable performance to models fully supervised with 3D labels, underscoring the significance of this approach in real-world applications.

Keywords: Deep Learning for Visual Perception , Computer Vision for Transportation , Visual Learning
Abstract: There are increasing interests of studying the video-to-depth (V2D) problem with machine learning techniques. While earlier methods directly learn a mapping from images to depth maps and camera poses, more recent works enforce multi-view geometry constraints through optimization embedded in the learning framework. This paper presents a novel optimization method based on recurrent neural networks to further

exploit the potential of neural networks in V2D. Specifically, our neural optimizer alternately updates the depth and camera poses through

iterations to minimize a feature-metric cost, and two gated recurrent units iteratively improve the results by tracing historical information. In this way, our network is a gradient-free zeroth-order optimizer designed for V2D and can be applied to both supervised and self-supervised V2D. Extensive experimental results demonstrate that our method outperforms previous methods and is more efficient in computation and memory consumption than cost-volume-based methods. In particular, our self-supervised method outperforms previous supervised methods on the KITTI and ScanNet datasets. Our source code will be made

public.

Keywords: Deep Learning for Visual Perception , Perception for Grasping and Manipulation , Visual Learning
Abstract: Dense visual correspondence plays a vital role in robotic perception. This work focuses on establishing the dense correspondence between a pair of images that captures dynamic scenes undergoing substantial transformations. We introduce Doduo to learn general dense visual correspondence from in-the-wild images and videos without ground truth supervision. Given a pair of images, it estimates the dense flow field encoding the displacement of each pixel in one image to its corresponding pixel in the other image. Doduo uses flow-based warping to acquire supervisory signals for the training. Incorporating semantic priors with self-supervised flow training, Doduo produces accurate dense correspondence robust to the dynamic changes of the scenes. Trained on an in-the-wild video dataset, Doduo illustrates superior performance on point-level correspondence estimation over existing self-supervised correspondence learning baselines. We also apply Doduo to articulation estimation and zero-shot goal-conditioned manipulation, underlining its practical applications in robotics.

Keywords: Deep Learning for Visual Perception , Recognition
Abstract: Robotic vision applications often necessitate a wide range of visual perception tasks, such as object detection, segmentation, and identification. While there have been substantial advances in these individual tasks, integrating specialized models into a unified vision pipeline presents significant engineering challenges and costs. Recently, Multimodal Large Language Models (MLLMs) have emerged as novel backbones for various downstream tasks. We argue that leveraging the pre-training capabilities of MLLMs enables the creation of a simplified framework, thus mitigating the need for task-specific encoders. Specifically, the large-scale pretrained knowledge in MLLMs allows for easier fine-tuning to downstream robotic vision tasks and yields superior performance. We introduce the RoboLLM framework, equipped with a BEiT-3 backbone, to address all visual perception tasks in the ARMBench challenge—a large-scale robotic manipulation dataset about real-world warehouse scenarios. RoboLLM not only outperforms existing baselines but also substantially reduces the engineering burden associated with model selection and tuning. All the code used in this paper can be found in https://github.com/longkukuhi/RoboLLM.

Keywords: Deep Learning for Visual Perception , RGB-D Perception , Visual Learning
Abstract: This paper introduces a novel approach named CrossVideo, which aims to enhance self-supervised cross-modal contrastive learning in the field of point cloud video understanding. Traditional supervised learning methods encounter limitations due to data scarcity and challenges in label acquisition. To address these issues, we propose a self-supervised learning method that leverages the cross-modal relationship between point cloud videos and image videos to acquire meaningful feature representations. Intra-modal and cross-modal contrastive learning techniques are employed to facilitate effective comprehension of point cloud video. We also propose a multi-level contrastive approach for both modalities. Through extensive experiments, we demonstrate that our method significantly surpasses previous state-of-the-art approaches, and we conduct comprehensive ablation studies to validate the effectiveness of our proposed designs.

Keywords: Deep Learning for Visual Perception , Visual Learning , RGB-D Perception
Abstract: Predicting accurate depth with monocular images is important for low-cost robotic applications and autonomous driving. This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes utilizing inter-frame poses obtained from inertial measurements. In particular, we introduce a Full-Scale depth prediction network named FSNet. FSNet contains four important improvements over existing self-supervised models: (1) a multichannel output representation for stable training of depth prediction in driving scenarios, (2) an optical-flow-based mask designed for dynamic object removal, (3) a self-distillation training strategy to augment the training process, and (4) an optimization-based post-processing algorithm in test time, fusing the results from visual odometry. With this framework, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data. Extensive experiments on the KITTI dataset, KITTI-360 dataset and the nuScenes dataset demonstrate the potential of FSNet. More visualizations are presented in url{https://sites.google.com/view/fsnet/home}

Keywords: Deep Learning for Visual Perception , Visual Learning , Simulation and Animation
Abstract: Dynamic Vision Sensor (DVS)-based solutions have recently garnered significant interest across various computer vision tasks, offering notable benefits in terms of dynamic range, temporal resolution, and inference speed. However, as a relatively nascent vision sensor compared to Active Pixel Sensor (APS) devices such as RGB cameras, DVS suffers from a dearth of ample labeled datasets. Prior efforts to convert APS data into events often grapple with issues such as a considerable domain shift from real events, the absence of quantified validation, and layering problems within the time axis. In this paper, we present a novel method for video-to-events stream conversion from multiple perspectives, considering the specific characteristics of DVS. A series of carefully designed losses helps enhance the quality of generated event voxels significantly. We also propose a novel local dynamic-aware timestamp inference strategy to accurately recover event timestamps from event voxels in a continuous fashion and eliminate the temporal layering problem. Results from rigorous validation through quantified metrics at all stages of the pipeline establish our method unquestionably as the current state-of-the-art (SOTA). The code can be found at bit.ly/v2ce.

Keywords: Data Sets for Robotic Vision , Deep Learning for Visual Perception , Big Data in Robotics and Automation
Abstract: Recent advances in vision-language models (VLMs) have led to improved performance on tasks such as visual question answering and image captioning. Consequently, these models are now well-positioned to reason about the physical world, particularly within domains such as robotic manipulation. However, current VLMs are limited in their understanding of the physical concepts (e.g., material, fragility) of common objects, which restricts their usefulness for robotic manipulation tasks that involve interaction and physical reasoning about such objects. To address this limitation, we propose PhysObjects, an object-centric dataset of 39.6K crowd-sourced and 417K automated physical concept annotations of common household objects. We demonstrate that fine-tuning a VLM on PhysObjects improves its understanding of physical object concepts, including generalization to held-out concepts, by capturing human priors of these concepts from visual appearance. We incorporate this physically grounded VLM in an interactive framework with a large language model-based robotic planner, and show improved planning performance on tasks that require reasoning about physical object concepts, compared to baselines that do not leverage physically grounded VLMs. We additionally illustrate the benefits of our physically grounded VLM on a real robot, where it improves task success rates. We release our dataset and provide further details and visualizations of our results at https://iliad.stanford.edu/pg-vlm/.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation
Abstract: Neural Radiance Fields (NeRFs) are gaining significant interest for online active object reconstruction due to their exceptional memory efficiency and requirement for only posed RGB inputs. Previous NeRF-based view planning methods exhibit computational inefficiency since they rely on an iterative paradigm, consisting of (1) retraining the NeRF when new images arrive; and (2) planning a path to the next best view only. To address these limitations, we propose a non-iterative pipeline based on the Prediction of the Required number of Views (PRV). The key idea behind our approach is that the required number of views to reconstruct an object depends on its complexity. Therefore, we design a deep neural network, named PRVNet, to predict the required number of views, allowing us to tailor the data acquisition based on the object complexity and plan a globally shortest path. To train our PRVNet, we generate supervision labels using the ShapeNet dataset. Simulated experiments show that our PRV-based view planning method outperforms baselines, achieving good reconstruction quality while significantly reducing movement cost and planning time. We further justify the generalization ability of our approach in a real-world experiment.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation
Abstract: Active object reconstruction using autonomous robots is gaining great interest. A primary goal in this task is to maximize the information of the object to be reconstructed, given limited on-board resources. Previous view planning methods exhibit inefficiency since they rely on an iterative paradigm based on explicit representations, consisting of (1) planning a path to the next-best view only; and (2) requiring a considerable number of less-gain views in terms of surface coverage. To address these limitations, we propose to integrate implicit representations into the One-Shot View Planning (OSVP). The key idea behind our approach is to use implicit representations to obtain the small missing surface areas instead of observing them with extra views. Therefore, we design a deep neural network, named OSVP, to directly predict a set of views given a dense point cloud refined from an initial sparse observation. To train our OSVP network, we generate supervision labels using dense point clouds refined by implicit representations and set covering optimization problems. Simulated experiments show that our method achieves sufficient reconstruction quality, outperforming several baselines under limited view and movement budgets. We further demonstrate the applicability of our approach in a real-world object reconstruction scenario.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , Sensor Fusion
Abstract: 3D object detection models that exploit both LiDAR and camera sensor features are top performers in large-scale autonomous driving benchmarks. A transformer is a popular network architecture used for this task, in which so-called object queries act as candidate objects. Initializing these object queries based on current sensor inputs is a common practice. For this, existing methods strongly rely on LiDAR data however, and do not fully exploit image features. Besides, they introduce significant latency. To overcome these limitations we propose EfficientQ3M, an efficient, modular, and multimodal solution for object query initialization for transformer-based 3D object detection models. The proposed initialization method is combined with a "modality-balanced" transformer decoder where the queries can access all sensor modalities throughout the decoder. In experiments, we outperform the state of the art in transformer-based LiDAR object detection on the competitive nuScenes benchmark and showcase the benefits of input-dependent multimodal query initialization, while being more efficient than the available alternatives for LiDAR-camera intitialization. The proposed method can be applied with any combination of sensor modalities as input, demonstrating its modularity.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , Simulation and Animation
Abstract: Reconstructing real-world objects and estimating their movable joint structures are pivotal technologies within the field of robotics. Previous research has predominantly focused on supervised approaches, relying on extensively annotated datasets to model articulated objects within limited categories. However, this approach falls short of effectively addressing the diversity present in the real world. To tackle this issue, we propose a self-supervised interaction perception method, referred to as SM3, which leverages multi-view RGB images captured before and after interaction to model articulated objects, identify movable parts, and infer the parameters of their rotating joints. By constructing 3D geometries and textures from the captured 2D images, SM3 achieves integrated optimization of movable parts and joint parameters during the reconstruction process, obviating the need for annotations. Furthermore, we introduce the MMArt dataset, an extension of PartNet-Mobility, encompassing multi-view and multi-modal data of articulated objects spanning diverse categories. Evaluations demonstrate that UM3 surpasses existing benchmarks across various categories and objects, while its adaptability in real-world scenarios has been duly validated.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , SLAM
Abstract: Moving object segmentation (MOS) provides a reliable solution for detecting traffic participants and thus is of great interest in the autonomous driving field. Dynamic capture is always critical in the MOS problem. While previous methods capture motion features from the range images directly, we argue that the residual maps provide greater potential for motion information, and on the other hand, range images contain rich semantic guidance. Based on this intuition, we propose MF-MOS, a novel motion-focused model with a dual-branch structure for Lidar moving object segmentation. Novelly, we decouple the spatial-temporal information by capturing the motion from residual maps and generating semantic features from range images, which are used as movable object guidance for the motion branch. Our straightforward yet distinctive solution can make the most use of both range images and residual maps, thus greatly improving the performance of the Lidar-based MOS task. Remarkably, our MF-MOS achieved a leading IoU of 76.7% on the MOS leaderboard of the SemanticKITTI dataset upon submission, demonstrating the current state-of-the-art performance. The implementation of our MF-MOS has been released at https://github.com/SCNU-RISLAB/MF-MOS.

Keywords: Deep Learning for Visual Perception , Computer Vision for Automation , Visual Learning
Abstract: 3D scene reconstruction from 2D images has been a long-standing task. Instead of estimating per-frame depth maps and fusing them in 3D, recent research leverages the neural implicit surface as a unified representation for 3D reconstruction. Equipped with data-driven pre-trained geometric cues, these methods have demonstrated promising performance. However, inaccurate prior estimation, which is usually inevitable, can lead to suboptimal reconstruction quality, particularly in some geometrically complex regions. In this paper, we propose a two-stage training process, decouple view-dependent and view-independent colors, and leverage two novel consistency constraints to enhance detail reconstruction performance without requiring extra priors. Additionally, we introduce an essential mask scheme to adaptively influence the selection of supervision constraints, thereby improving performance in a self-supervised paradigm. Experiments on synthetic and real-world datasets show the capability of reducing the interference from prior estimation errors and achieving high-quality scene reconstruction with rich geometric details.

Keywords: Deep Learning for Visual Perception , Deep Learning in Grasping and Manipulation , Grasping
Abstract: We present Normal Field Learning (NFL), a robust yet practical solution to perceive 3D layouts of transparent objects and grasp them quickly. Conventional input modalities for vision-based grasping do not provide sufficient information for transparent objects. However, with the recent advance on datasets and algorithms for transparent objects, we can at least obtain noisy estimates of normals and masks for various real-world conditions. Instead of directly using the RGB images, we propose to use the estimates to train a neural volume, which serves as an intermediate representation ignorant of challenging appearance variations. We formulate the training objective to account for in- herent uncertainty in individual estimation, and together with the volumetric aggregation, we can reliably extract useful geometric information for grasping. Our neural volume deploys a voxel- grid based representation, motivated by acceleration techniques of neural radiance fields. However, we directly store the normal and density values in the grid cells instead of latent features. Our modification allows direct access to the geometric values without additional inference or volume rendering, further enhancing the efficiency. Our results show over 85% success rates in grasping in cluttered scenes with only 40 seconds of training time.

Keywords: Deep Learning for Visual Perception , Perception for Grasping and Manipulation , Data Sets for Robotic Vision
Abstract: Precise reconstruction and manipulation of the crumpled cloths is challenging due to the high dimensionality of cloth models, as well as the limited observation at self-occluded regions. We leverage the recent progress in the field of single-view human reconstruction to template-based reconstruct crumpled cloths from their top-view depth observations only, with our proposed sim-real registration protocols. In contrast to previous implicit cloth representations, our reconstruction mesh explicitly describes the positions and visibilities of the entire cloth mesh vertices, enabling more efficient dual-arm and single-arm target-oriented manipulations. Experiments demonstrate that our TRTM system can be applied to daily cloths that have similar topologies as our template mesh, but with different shapes, sizes, patterns, and physical properties. Videos, datasets, pre-trained models, and code can be downloaded from our project website: https://wenbwa.github.io/TRTM/.

Keywords: Model Learning for Control , Aerial Systems: Mechanics and Control , Learning and Adaptive Systems , Optimization and Optimal Control
Abstract: Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the learned dynamics. This controller actively chooses the optimal control actions that (i) optimize the control performance and (ii) improve the efficiency of online learning sample collection. We demonstrate the effectiveness of our method through a series of cha

Keywords: Model Learning for Control , AI-Based Methods , Dexterous Manipulation
Abstract: Deformable object manipulation presents a unique set of challenges in robotic manipulation by exhibiting high degrees of freedom and severe self-occlusion. State representation for materials that exhibit plastic behavior, like modeling clay or bread dough, is also difficult because they permanently deform under stress and are constantly changing shape. In this work, we investigate each of these challenges using the task of robotic sculpting with a parallel gripper. We propose a system that uses point clouds as the state representation and leverages pre-trained point cloud reconstruction Transformer to learn a latent dynamics model to predict material deformations given a grasp action. We design a novel action sampling algorithm that reasons about geometrical differences between point clouds to further improve the efficiency of model-based planners. All data and experiments are conducted entirely in the real world. Our experiments show the proposed system is able to successfully capture the dynamics of clay, and is able to create a variety of simple shapes. Videos and additional figures are available on our project page at: https://sites.google.com/andrew.cmu.edu/sculptbot

Keywords: Model Learning for Control , Autonomous Agents , Machine Learning for Robot Control
Abstract: Advancements in Automated Driving Systems (ADSs) have enabled the achievement of a certain level of autonomy while commuting in a car. However, emergency and high-speed maneuvers still arise as significant challenges for ADSs due to the intrinsic nonlinearity and fast-paced behavior of such events. These maneuvers are a distinctive feature within the recently established motorsport discipline of Autonomous Racing (AR). In this work, we explore the use of Learning-based Model Predictive Control (LMPC) to address possible model mismatches of the first principles model in high-speed racing. To this end, a Model Predictive Contouring Control (MPCC) (a specific formulation of the standard Model Predictive Control, MPC) is formulated, and a Neural Network (NN) that leverages the use of Feedforward and Recurrent layers is employed to learn the errors of the first principles model. By combining the NN with the first principles model, the LMPC is born, capable of accurately predicting the future with a computational effort compatible with real-time feasibility, effectively handling the vehicle at its limits. Furthermore, the controller can adapt to changing environments by training the NN during the race. The MPCC (formulation without the NN) is deployed on a real autonomous formula student racing car showing an improvement of 16% in mean lap times across the same track between a common geometric controller. The LMPC is analyzed in a high-fidelity simulator, achieving an improvement of 8.9% in mean lap times when compared to the MPCC.

Keywords: Model Learning for Control , Autonomous Vehicle Navigation , Field Robots
Abstract: High-speed autonomous driving in off-road environments has immense potential for various applications, but it also presents challenges due to the complexity of vehicle-terrain interactions. In such environments, it is crucial for the vehicle to predict its motion and adjust its controls proactively in response to environmental changes, such as variations in terrain elevation. To this end, we propose a method for learning terrain-aware kinodynamic model which is conditioned on both proprioceptive and exteroceptive information. The proposed model generates reliable predictions of 6-degree-of-freedom motion and can even estimate contact interactions without requiring ground truth force data during training. This enables the design of a safe and robust model predictive controller through appropriate cost function design which penalizes sampled trajectories with unstable motion, unsafe interactions, and high levels of uncertainty derived from the model. We demonstrate the effectiveness of our approach through experiments on a simulated off-road track, showing that our proposed model-controller pair outperforms the baseline and ensures robust high-speed driving performance without control failure.

Keywords: Model Learning for Control , Calibration and Identification , Optimization and Optimal Control
Abstract: Robotic adaptation to unanticipated operating conditions is crucial to achieving persistence and robustness in complex real world settings. For a wide range of cutting-edge robotic systems, such as micro- and nano-scale robots, soft robots, medical robots, and bio-hybrid robots, it is infeasible to anticipate the operating environment a priori due to complexities that arise from numerous factors including imprecision in manufacturing, chemo-mechanical forces, and poorly understood contact mechanics. Drawing inspiration from data-driven modeling, geometric mechanics (or gauge theory), and adaptive control, we employ an adaptive system identification framework and demonstrate its efficacy in enhancing the performance of principally kinematic locomotors (those governed by Rayleigh dissipation or zero momentum conservation). We showcase the capability of the adaptive model to efficiently accommodate varying terrains and iteratively modified behaviors within a behavior optimization framework. This provides both the ability to improve fundamental behaviors and perform motion tracking to precision. Notably, we are capable of optimizing the gaits of the Purcell swimmer using approximately 10 cycles per link, which for the nine-link Purcell swimmer provides a factor of ten improvement in optimization speed over the state of the art. Beyond simply a computational speed up, this ten-fold improvement may enable this method to be successfully deployed for in-situ behavior refinement, injury recovery, and terrain adaptation, particularly in domains where simulations provide poor guides for the real world.

Keywords: Model Learning for Control , Calibration and Identification , Robust/Adaptive Control
Abstract: This study presents a recursive algorithm for solving the regularised least squares problem for online identification of rigid body dynamic model parameters with emphasis on the physical consistency of estimated inertial parameters. One of the geometric approaches is to use a regulariser that represents how close the pseudo-inertia matrix is to a given reference on the feasible manifold in the regression problem. The proposed extension enables memory-efficient online learning in addition to the benefits of geometry-aware convex regularisation using the log-determinant divergence of the pseudo-inertia matrix. Also, the resursive version endows the estimator with the capability to deal with time-variation of parameters by introducing an optional forgetting mechanism. The characteristics of the recursive regularised least squares algorithm is demonstrated using the MIT Cheetah 3 leg swinging experiment dataset and compared to the existing batch optimisation method.

Keywords: Model Learning for Control , Compliant Assembly , Robust/Adaptive Control
Abstract: Robotic manipulation of deformable linear objects (DLOs) is an active area of research, though emerging applications, like automotive wire harness installation, introduce constraints that have not been considered in prior work. Confined workspaces and limited visibility complicate prior assumptions of multi-robot manipulation and direct measurement of DLO configuration (state). This work focuses on single-arm manipulation of stiff DLOs (StDLOs) connected to form a DLO network (DLON), for which the measurements (output) are the endpoint poses of the DLON, which are subject to unknown dynamics during manipulation. To demonstrate feasibility of output-based control without state estimation, direct input-output dynamics are shown to exist by training neural network models on simulated trajectories. Output dynamics are then approximated with polynomials and found to contain well-known rigid body dynamics terms. A composite model consisting of a rigid body model and an online data-driven residual is developed, which predicts output dynamics more accurately than either model alone, and without prior experience with the system. An adaptive model predictive controller is developed with the composite model for DLON manipulation, which completes DLON installation tasks, both in simulation and with a physical automotive wire harness.

Keywords: Model Learning for Control , Deep Learning Methods , Aerial Systems: Applications
Abstract: Recent advances in aerial robotics have enabled the use of multirotor vehicles for autonomous payload transportation. Resorting only to classical methods to reliably model a quadrotor carrying a cable-slung load poses significant challenges. On the other hand, purely data-driven learning methods do not comply by design with the problem's physical constraints, especially in states that are not densely represented in training data. In this work, we explore the use of physics-informed neural networks to learn an end-to-end model of the multirotor-slung-load system and, at a given time, estimate a sequence of the future system states. An LSTM encoder-decoder with an attention mechanism is used to capture the dynamics of the system. To guarantee the cohesiveness between the multiple predicted states of the system, we propose the use of a physics-based term in the loss function, which includes a discretized physical model derived from first principles together with slack variables that allow for a small mismatch between expected and predicted values. To train the model, a dataset using a real-world quadrotor carrying a slung load was curated and is made available. Prediction results are presented and corroborate the feasibility of the approach. The proposed method outperforms both the first principles physical model and a comparable neural network model trained without the physics regularization proposed.

Keywords: Model Learning for Control , Dynamics , Machine Learning for Robot Control
Abstract: Skid-Steer Wheeled Mobile Robots (SSWMRs) are increasingly being used for off-road autonomy applications. When turning at high speeds, these robots tend to undergo significant skidding and slipping. In this work, using Gaussian Process Regression (GPR) and Sigma-Point Transforms, we estimate the non-linear effects of tire-terrain interaction on robot velocities in a probabilistic fashion. Using the mean estimates from GPR, we propose a data-driven dynamic motion model that is more accurate at predicting future robot poses than conventional kinematic motion models. By efficiently solving a convex optimization problem based on the history of past robot motion, the GPR augmented motion model generalizes to previously unseen terrain conditions. The output distribution from the proposed motion model can be used for local motion planning approaches, such as stochastic model predictive control, leveraging model uncertainty to make safe decisions. We validate our work on a benchmark real-world multi-terrain SSWMR dataset. Our results show that the model generalizes to three different terrains while significantly reducing errors in linear and angular motion predictions. As shown in the attached video, we perform a separate set of experiments on a physical robot to demonstrate the robustness of the proposed algorithm.

Keywords: Data Sets for Robot Learning , Field Robots , Big Data in Robotics and Automation
Abstract: We present TartanDrive 2.0, a large-scale off-road driving dataset for self-supervised learning tasks. In 2021 we released TartanDrive 1.0, which is one of the largest datasets for off-road terrain. As a follow up to our original dataset, we collected seven hours of data at speeds of up to 15m/s with the addition of three new LiDAR sensors alongside the original camera, inertial, GPS, and proprioceptive sensors. We also release the tools we use for collecting, processing, and querying the data, including our metadata system designed to further the utility of our data. Custom infrastructure allows end users to reconfigure the data to cater to their own platforms. These tools and infrastructure alongside the dataset are useful for a variety of tasks in the field of off-road autonomy and, by releasing them, we encourage collaborative data aggregation. These resources lower the barrier to entry to utilizing large-scale datasets, thereyby helping facilitate the advancement of robotics in areas such as self-supervised learning, multi-modal perception, inverse reinforcement learning, and representation learning. The dataset is available at https://theAirLab.org/TartanDrive2.

Keywords: Deep Learning Methods , Marine Robotics , Object Detection, Segmentation and Categorization
Abstract: In recent years, significant progress has been made in the field of underwater image enhancement (UIE). However, its practical utility for high-level vision tasks, such as underwater object detection (UOD) in Autonomous Underwater Vehicles (AUVs), remains relatively unexplored. It may be attributed to several factors: (1) Existing methods typically employ UIE as a pre-processing step, which inevitably introduces considerable computational overhead and latency. (2) The process of enhancing images prior to training object detectors may not necessarily yield performance improvements. (3) The complex underwater environments can induce significant domain shifts across different scenarios, seriously deteriorating the UOD performance. To address these challenges, we introduce EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capabilities. Specifically, both the UIE and UOD task heads share the same network backbone and utilize a lightweight design. Furthermore, to ensure balanced training for both tasks, we present a multi-stage training strategy aimed at consistently enhancing the performance of both functions. Additionally, we propose a novel domain-adaptation strategy to align feature embeddings originating from diverse underwater environments. Comprehensive experiments demonstrate that our framework not only achieves state-of-the-art (SOTA) performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios. Our efficiency analysis further highlights the substantial potential of our framework for onboard deployment.

Keywords: Field Robots , Deep Learning Methods , Simulation and Animation
Abstract: To avoid unmanned ground vehicles being obstructed by deformed terrain in off-road, effective vehicle mobility analysis is required. However, the computational complexity of existing mobility analysis methods, such as discrete element analysis, poses significant challenges when applied to largescale terrains. To address this problem, we propose an efficient and high-fidelity vehicle mobiliy prediction method for a largescale terrain. Initially, precise terrain models are constructed employing Gaussian sampling (GS), thereby serving as optimal inputs for the mobility simulation. Subsequently, we introduce a co-simulation method based on a multi-body dynamics model and discrete element analysis to obtain high-fidelity vehicle mobility data on sampled terrains. Following that, the mobility data is utilized to train a PSO-kriging neural network (PKNN), enabling accurate predictions of the global mobility map. Through rigorous simulation experiments, the proposed method (GS-PKNN) demonstrates its remarkable effectiveness.

Keywords: Field Robots , Legged Robots , Deep Learning Methods
Abstract: Traversability estimation in rugged, unstructured environments remains a challenging problem in field robotics. Often, the need for precise, accurate traversability estimation is in direct opposition to the limited sensing and compute capability present on affordable, small-scale mobile robots. To address this issue, we present a novel method to learn [u]ncertainty-aware [n]avigation features from high-fidelity scans of [real]-world environments (UNRealNet). This network can be deployed on-robot to predict these high-fidelity features using input from lower-quality sensors. UNRealNet predicts dense, metric-space features directly from single-frame lidar scans, thus reducing the effects of occlusion and odometry error. Our approach is label-free, and is able to produce traversability estimates that are robot-agnostic. Additionally, we can leverage UNRealNet’s predictive uncertainty to both produce risk-aware traversability estimates, and refine our feature predictions over time. We find that our method outperforms traditional local mapping and inpainting baselines by up to 40%, and demonstrate its efficacy on multiple legged platforms.

Keywords: Field Robots , Motion and Path Planning , Data Sets for Robot Learning
Abstract: Efficient and reliable navigation in off-road environments poses a significant challenge for robotics, especially when factoring in the varying capabilities of robots across different terrains. To achieve this, the robot system's traversability is usually estimated to plan traversable routes through an environment. This paper presents a new approach that utilizes Deep Multimodal Variational Autoencoders (DMVAEs) for estimating the traversability of different robots in complex off-road terrains. Our method utilizes DMVAEs to capture essential environmental information and robot properties, effectively modeling factors that influence robotic traversability. The key contribution of this research is a two-stage traversability estimation framework for various robots in diverse off-road conditions that integrates robot properties in addition to environmental information to predict the traversability for various robots in a single model. We validate our method through real-world experiments involving four ground robots navigating an alpine environment. Comparative evaluations against state-of-the-art traversability estimation methods demonstrate the superior accuracy and robustness of our approach. Additionally, we investigate the transfer of trained models to new robots, enhancing their traversability estimation and extending the applicability of our framework.

Keywords: Field Robots , Motion and Path Planning , Deep Learning Methods
Abstract: In wild environments, motion planning for mobile robots faces the challenge of local optimal path traps due to limited sensor perception range and lack of spatial awareness. Existing approaches that avoid local optimum by designing heuristic functions or high-quality global paths in wild environments are time-consuming and unstable. This work proposes F3DMP, which consists of two parts to alleviate the local optimum solution and better utilize distant terrain information. First, the entire planning framework is adapted to the three-dimensional space so that the planning result conforms to the geometric characteristics of the terrain. Second, a time allocation function based on offline reinforcement learning is proposed. This function can anticipate potential challenges or opportunities based on semantic information for the image and proactively determine a time allocation. Our planner is integrated into a complete mobile robot system and deployed to a real robot. Experiments in simulation and the real world demonstrate that our method can improve the success rate by 28% and the trajectory smoothness by 27% compared with traditional methods.

Keywords: Intelligent Transportation Systems , Path Planning for Multiple Mobile Robots or Agents , Datasets for Human Motion
Abstract: Data-driven methods have great advantages in modeling complicated human behavioral dynamics and dealing with many human-robot interaction applications. However, collecting massive and annotated real-world human datasets has been a laborious task, especially for highly interactive scenarios. On the other hand, algorithmic data generation methods are usually limited by their model capacities, making them unable to offer realistic and diverse data needed by various application users. In this work, we study trajectory-level data generation for multi-human or human-robot interaction scenarios and propose a learning-based automatic trajectory generation model, which we call Multi-Agent TRajectory generation with dIverse conteXts (MATRIX). MATRIX is capable of generating interactive human behaviors in realistic diverse contexts. We achieve this goal by modeling the explicit and interpretable objectives so that MATRIX can generate human motions based on diverse destinations and heterogeneous behaviors. We carried out extensive comparison and ablation studies to illustrate the effectiveness of our approach across various metrics. We also presented experiments that demonstrate the capability of MATRIX to serve as data augmentation for imitation-based motion planning.

Keywords: Agricultural Automation , Planning, Scheduling and Coordination
Abstract: In multi-robot informative path planning the problem is to find a route for each robot in a team to visit a set of locations that can provide the most useful data to reconstruct an unknown scalar field. In the budgeted version, each robot is subject to a travel budget limiting the distance it can travel. Our interest in this problem is motivated by applications in precision agriculture, where robots are used to collect measurements to estimate domain-relevant scalar parameters such as soil moisture or nitrates concentrations. In this paper, we propose an online, distributed multi-robot sampling algorithm based on Monte Carlo Tree Search (MCTS) where each robot iteratively selects the next sampling location through communication with other robots and considering its remaining budget.

We evaluate our proposed method for varying team sizes and in different environments, and we compare our solution with four different baseline methods. Our experiments show that our solution outperforms the baselines when the budget is tight by collecting measurements leading to smaller reconstruction errors.

Keywords: Agricultural Automation , Planning, Scheduling and Coordination , Task Planning
Abstract: A mobile manipulator often finds itself in an application where it needs to take a close-up view before performing a manipulation task. Named this as a coupled active perception and manipulation (CAPM) problem, we model the uncertainty in the perception process and devise a key state/task planning approach that considers reachability conditions as task constraints of both perception and manipulation tasks for the mobile platform. By minimizing the expected energy usage in body key state planning while satisfying task constraints, our algorithm achieves the best balance between the task success rate and energy usage. We have implemented the algorithm and tested it in both simulation and physical experiments. The results have confirmed that our algorithm has a lower energy consumption compared to a two-stage decoupled approach, while still maintaining a success rate of 100% for the task.

Keywords: Performance Evaluation and Benchmarking , Assembly , Task Planning
Abstract: We introduce RAMP, an open-source robotics benchmark inspired by real-world industrial assembly tasks. RAMP consists of beams that a robot must assemble into specified goal configurations using pegs as fasteners. As such it assesses planning and execution capabilities, and poses challenges in perception, reasoning, manipulation, diagnostics, fault recovery, and goal parsing. RAMP has been designed to be accessible and extensible. Parts are either 3D printed or otherwise constructed from materials that are readily obtainable. The part design and detailed instructions are publicly available. In order to broaden community engagement, RAMP incorporates fixtures such as April Tags which enable researchers to focus on individual sub-tasks of the assembly challenge if desired. We provide a full digital twin as well as rudimentary baselines to enable rapid progress. Our vision is for RAMP to form the substrate for a community-driven endeavour that evolves as capability matures.

Keywords: Safety in HRI , Human-Centered Robotics , Task Planning
Abstract: The use of pointed or edged tools or objects is one of the most challenging aspects of today's application of physical human-robot interaction (pHRI). One reason for this is that the severity of harm caused by such edged or pointed impactors is less well studied than for blunt impactors. Consequently, the standards specify well-reasoned force and pressure thresholds for blunt impactors and advise avoiding any edges and corners in contacts. Nevertheless, pointed or edged impactor geometries cannot be completely ruled out in real pHRI applications. For example, to allow edged or pointed tools such as screwdrivers near human operators, the knowledge of injury severity needs to be extended so that robot integrators can perform well-reasoned, time-efficient risk assessments. In this paper, we provide the initial datasets on injury prevention for the human hand based on drop tests with surrogates for the human hand, namely pig claws and chicken drumsticks. We then demonstrate the ease and efficiency of robot use using the dataset for contact on two examples. Finally, our experiments provide a set of injuries that may also be expected for human subjects under certain robot mass-velocity constellations in collisions. To extend this work, testing on human samples and a collaborative effort from research institutes worldwide is needed to create a comprehensive human injury avoidance database for any pHRI scenario and thus for safe pHRI applications including edged and pointed geometries.

Keywords: Task Planning , AI-Enabled Robotics , Learning from Experience
Abstract: Long-horizon task planning is important for robot autonomy, especially as a subroutine for frameworks such as Integrated Task and Motion Planning. However, task planning is computationally challenging and struggles to scale to realistic problem settings. We propose to accelerate task planning over an agent's lifetime by integrating abstract strategies: a generalizable planning experience encoding introduced in earlier work. In this work, we contribute a practical approach to planning with strategies by introducing a novel formalism of planning in a strategy-augmented domain. We also introduce and formulate the notion of a strategy's affordance, which indicates its predicted benefit to the solution, and use it to guide the planning and strategy grounding processes. Together, our observations yield an affordance-directed, lazy-search planning algorithm, which can seamlessly compose strategies and actions to solve long-horizon planning problems. We evaluate our planner in an object rearrangement domain, where we demonstrate performance benefits relative to a state-of-the-art task planner.

Keywords: Task Planning , Learning from Demonstration
Abstract: Following work on joint object-action representations, functional object-oriented networks (FOON) were introduced as a knowledge graph representation for robots. A FOON contains symbolic concepts useful to a robot’s understanding of tasks and its environment for object-level planning. Prior to this work, little has been done to show how plans acquired from FOON can be executed by a robot, as the concepts in a FOON are too abstract for execution. We thereby introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution. Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills in a hierarchical planning pipeline to generate executable task plans. We demonstrate our entire approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.

Keywords: Task Planning , Manipulation Planning , AI-Enabled Robotics
Abstract: Task planning for robotic cooking involves generating a sequence of actions for a robot to prepare a meal successfully. This paper introduces a novel task tree generation pipeline producing correct planning and efficient execution for cooking tasks. Our method first uses a large language model (LLM) to retrieve recipe instructions and then utilizes a fine-tuned GPT-3 to convert them into a task tree, capturing sequential and parallel dependencies among subtasks. The pipeline then mitigates the uncertainty and unreliable features of LLM outputs using task tree retrieval. We combine multiple LLM task tree outputs into a graph and perform a task tree retrieval to avoid questionable nodes and high-cost nodes to improve planning correctness and improve execution efficiency. Our evaluation results show its superior performance compared to previous works in task planning accuracy and efficiency.

Keywords: Task Planning , Multi-Robot Systems , Autonomous Agents
Abstract: This paper presents a novel stepwise multi-agent task planning method that incorporates neighborhood search to address large-scale problems, thereby reducing computation time. With an increasing number of agents,

the search space for task planning expands exponentially. Hence, conventional methods aiming to find globally optimal solutions, especially for some large-scale problems, incur extremely high computational costs and may even fail. In this paper, the proposed method easily achieves the goals of multi-agent task planning by solving an initial problem using a minimal number of agents. Subsequently, tasks are reallocated among all agents based on this solution and the solutions are iteratively optimized using a neighborhood search. While aiming to find a near-optimal solution rather than an optimal one, the method substantially reduces the time complexity of searching to a polynomial level. Moreover, the effectiveness of the proposed method is demonstrated by solving some benchmark problems and comparing the results obtained using the proposed method with those obtained using other state-of-the-art methods.

Keywords: Task Planning , Multi-Robot Systems , Multi-Robot SLAM
Abstract: With the increasing demand for multi-robot exploration of unknown environments, how to accomplish this problem efficiently has become a focus of research. However, in this kind of task, the formulation of strategies for frontier point detection and task allocation largely determines the overall efficiency of the system. In the task of multi-robot exploration of unknown environments, the strategies of frontier point detection and task assignment determine the overall efficiency of the system. Most of the existing methods implement frontier point detection based on the Rapidly-Exploring Random Tree (RRT) and use greedy algorithms for task allocation. However, the classical RRT algorithm is a fixed growth step, which leads to the difficulty of growing branches in narrow environments, making the efficiency and correctness of detecting frontier points lower. Meanwhile, the allocation strategy of the greedy algorithm causes each robot to consider only the exploration area with the largest gain for itself, which easily leads to repeated exploration and reduces the overall efficiency of the system. To solve these problems, we propose an adaptive RRT tree growth strategy for frontier point detection, which can adjust the step size according to the known map information and thus improve the efficiency and accuracy of detection; and introduce a Bayesian-guided evolutionary strategy(BGE) for efficient task allocation, which can utilize the current and historical information to find the optimal allocation scheme in a global perspective. We conduct a comprehensive test of the proposed strategy in the ROS system as well as in the real world, which proves the efficiency of our strategy.

Keywords: Modeling, Control, and Learning for Soft Robots , Compliance and Impedance Control
Abstract: Continuum robots with variable stiffness have gained wide popularity in the last decade. Layer jamming (LJ) has emerged as a simple and efficient technique to achieve tunable stiffness for continuum robots. Despite its merits, the development of a control-oriented dynamical model ¹ tailored for this specific class of robots remains an open problem in the literature. This paper aims to present the first solution, to the best of our knowledge, to close the gap. We propose an energy-based model that is integrated with the LuGre frictional model for LJ-based continuum robots. Then, we take a comprehensive theoretical analysis for this model, focusing on two fundamental characteristics of LJ-based continuum robots: shape locking and adjustable stiffness. To validate the modeling approach and theoretical results, a series of experiments using our OctRobot-I continuum robotic platform was conducted. The results show that the proposed model is capable of interpreting and predicting the dynamical behaviors in LJ-based continuum robots.

Keywords: Modeling, Control, and Learning for Soft Robots , Dynamics , Medical Robots and Systems
Abstract: This paper presents a lumped-parameter dynamic model of a pressure driven eversion robot carrying a catheter through its hollow core. A simulation framework based on the model is developed in MATLAB and is used for understanding the underlying physics, for identifying the regions of operation, and for demonstrating that, for a range of input commands, the catheter can be used as an actuation mechanism for propelling eversion; an approach especially useful for miniaturised systems. Simulations are experimentally validated on the MAMMOBOT system, which is a miniature steerable soft growing robot for early breast cancer detection. It was demonstrated that for most regions of operation experimental results compare well with simulation exhibiting an error less than 4%. Only one region of operation demonstrated larger deviations due possibly to unmodeled dynamics, which will be investigated in future work.

Keywords: Modeling, Control, and Learning for Soft Robots , Hydraulic/Pneumatic Actuators , Robust/Adaptive Control
Abstract: Soft robotics is an emergent and swiftly evolving field. Pneumatic actuators are suitable for driving soft robots because of their superior performance. However, their control is not easy due to their hysteresis characteristics. In response to these challenges, we propose

an adaptive control method to compensate hysteresis of a soft actuator.

Employing a novel dual pneumatic artificial muscle (PAM) bending actuator, the innovative control strategy abates hysteresis effects by dynamically modulating gains within a traditional PID controller corresponding with the predicted motion of the reference trajectory. Through comparative experimental evaluation, we found that the new control method outperforms its conventional counterparts regarding tracking accuracy and response speed. Our work reveals a new direction for advancing control in soft actuators.

Keywords: Modeling, Control, and Learning for Soft Robots , Kinematics , Motion Control
Abstract: The passive compliance of soft robotic arms renders the development of accurate kinematic models and model-based controllers challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC introduces errors when the robot is subject to external forces or even gravity. In this paper, we establish a three-dimensional (3D) kinematic representation of a soft robotic arm with pseudo universal and prismatic joints that are capable of capturing non-constant curvature deformations of the soft segments. We theoretically demonstrate that this constitutes a more general methodology than PCC. Simulations and experiments on the real robot attest to the superior modeling accuracy of our approach in 3D motions with unknown load. The maximum position/rotation error of the proposed model is verified 6.7times/4.6times lower than the PCC model considering gravity and external forces. Furthermore, we devise an inverse kinematic controller that is capable of positioning the tip, tracking trajectories, as well as performing interactive tasks in the 3D space.

Keywords: Modeling, Control, and Learning for Soft Robots , Kinematics
Abstract: The small size, high dexterity, and intrinsic compliance of continuum robots (CRs) make them well suited for constrained environments. Solving the inverse kinematics (IK), that is finding robot joint configurations that satisfy desired position or pose queries, is a fundamental challenge in motion planning, control, and calibration for any robot structure. For CRs, the need to avoid obstacles in tightly confined workspaces greatly complicates the search for feasible IK solutions. Without an accurate initialization or multiple restarts, existing algorithms often fail to find a solution. We present CIDGIKc (Convex Iteration for Distance-Geometric Inverse Kinematics for Continuum Robots), an algorithm that solves these nonconvex feasibility problems with a sequence of semidefinite programs whose objectives are designed to encourage low-rank minimizers. CIDGIKc is enabled by a novel distance-geometric parameterization of constant curvature segment geometry for CRs with extensible segments. The resulting IK formulation involves only quadratic expressions and can efficiently incorporate a large number of collision avoidance constraints. Our experimental results demonstrate >98% solve success rates within complex, highly cluttered environments which existing algorithms cannot account for.

Keywords: Modeling, Control, and Learning for Soft Robots , Machine Learning for Robot Control
Abstract: Soft robots have desirable qualities for use in underwater environments

thanks to their inherent compliance and lack of need for exposed hardware. Nevertheless, these advantages come at the cost of considerable control challenges. Data-driven model predictive control (MPC) is an approach that has shown promise in controlling soft robots.

However, manually tuning the many hyperparameters in the learned dynamics model and the optimizer can be extremely tedious. In this work, we explore using data-driven MPC to control an underwater soft robot, and employ the AutoMPC method to automatically tune the hyperparameters and generate the controller. In the process, we extend AutoMPC’s capabilities to handle multi-task tuning and we add a barrier

cost function to enforce actuator constraints. Our experiments show that the AutoMPC controller reaches targets with significantly higher accuracy and reliability than state-of-the-art baselines both in- and out-of-distribution of the training data.

Keywords: Automation at Micro-Nano Scales , Micro/Nano Robots , Soft Robot Applications
Abstract: Acoustic techniques have been developed as multifunctional tools for various microscale manipulations. In prevalent design paradigms, a position-fixed piezoelectric transducer (PZT) is utilized to generate ultrasound waves. However, the immobility of the PZT restricts the modulation of the acoustic field's position and orientation, consequently diminishing the adaptability and effectiveness of subsequent acoustic micromanipulation tasks. Here, we proposed a miniaturized soft acoustic end-effector and demonstrated acoustic field modulation and microparticle manipulation by adjusting PZT position and orientation. The PZT is mounted on the end of a soft robotic arm that has three individual degrees of freedom and can be deformed in 3D space by inflating or deflating each chamber. Experiments showed that the soft acoustic end-effector can change the traveling direction of microparticles and modulate the location of a standing wave field. Our approach is simple, flexible, and controllable. We envision that the soft acoustic end-effector will facilitate multiscale acoustic manipulation in interdisciplinary applications, especially, for in vivo acoustic therapies.

Keywords: Modeling, Control, and Learning for Soft Robots , Motion Control
Abstract: Fully exploiting soft robots’ capabilities requires devising strategies that can accurately control their movements with the limited amount of control sources available. This task is challenging for reasons including the hard-to-model dynamics, the system’s underactuation, and the need of using a prominent feedforward control action to preserve the soft and safe robot behavior. To tackle this challenge, this letter proposes a purely feedforward iterative learning control algorithm that refines the torque action by leveraging both the knowledge of the model and data obtained from past experience. After presenting a 3D polynomial description of soft robots, we study their intrinsic properties, e.g., input-to-state stability, and we prove the convergence of the controller coping with locally Lipschitz nonlinearities. Finally, we validate the proposed approach through simulations and experiments involving multiple systems, trajectories, and in the case of external disturbances and model mismatches.

Keywords: Learning from Demonstration , Motion Control of Manipulators , Deep Learning in Robotics and Automation , Dynamical Systems as Movement Primitives
Abstract: Learning from humans allows non-experts to program robots with ease, lowering the resources required to build complex robotic solutions. Nevertheless, such data-driven approaches often lack the ability to provide guarantees regarding their learned behaviors, which is critical for avoiding failures and/or accidents. In this work, we focus on reaching/point-to-point motions, where robots must always reach their goal, independently of their initial state. This can be achieved by modeling motions as dynamical systems and ensuring that they are globally asymptotically stable. Hence, we introduce a novel Contrastive Learning loss for training Deep Neural Networks (DNN) that, when used together with an Imitation Learning loss, enforces the aforementioned stability in the learned motions. Differently from previous work, our method does not restrict the structure of its function approximator, enabling its use with arbitrary DNNs and allowing it to learn complex motions with high accuracy. We validate it using datasets and a real robot. In the former case, motions are 2 and 4 dimensional, modeled as first- and second-order dynamical systems. In the latter, motions are 3, 4, and 6 dimensional, of first and second order, and are used to control a 7DoF robot manipulator in its end effector space and joint space. More details regarding the real-world experiments are presented in: https://youtu.be/OM-2edHBRfc.

Keywords: Learning from Demonstration , Deep Learning Methods , Machine Learning for Robot Control
Abstract: Movement Primitive (MP) is a promising Learning from Demonstration (LfD) framework, which is commonly used to learn movements from human demonstrations and adapt the learned movements to new task scenes. A major goal of MP research is to improve the adaptability of MP to various target positions and obstacles. MPs enable their adaptability by capturing the variability of demonstrations. However, current MPs can only learn from low-variance demonstrations. The low-variance demonstrations include varied target positions but leave various obstacles alone. These MPs can not adapt the learned movements to the task scenes with different obstacles, which limits their adaptability since obstacles are everywhere in daily life. In this paper, we propose a novel transformer and GAN-based Editable Movement Primitive (EditMP), which can learn movements from high-variance demonstrations. These demonstrations include the movements in the task scenes with various target positions and obstacles. After movement learning, EditMP can controllably and interpretably edit the learned movements for new task scenes. Notably, EditMP enables all robot joints rather than the robot end-effector to avoid hitting complex obstacles. The proposed method is evaluated on three tasks and deployed to a real-world robot. We compare EditMP with probabilistic-based MPs and empirically demonstrate the state-of-the-art adaptability of EditMP.

Keywords: Learning from Demonstration , Imitation Learning , Task and Motion Planning
Abstract: The proposed work presents a framework based on Graph Neural Networks (GNN) that abstracts the task to be executed and directly allows the robot to learn task-specific rules from synthetic demonstrations given through imitation learning. A graph representation of the state space is considered to encode the task-relevant entities as nodes for a Pick-and-Place task declined at different levels of difficulty. During training, the GNN-based policy learns the underlying rules of the manipulation task focusing on the structural relevance and the type

of objects and goals, relying on an external primitive to move the robot to accomplish the task. The GNN-policy has been trained as a node-classification approach by looking at the different configurations

of the objects and goals present in the scene, learning the association

between them with respect to their type for the Pick-and-Place task. The experimental results show a high generalization capability of the proposed model in terms of the number, positions, height distributions,

and even configurations of the objects/goals. Thanks to the generalization, only a single image of the desired goal configuration is required at inference time.

Keywords: Learning from Demonstration , Compliance and Impedance Control
Abstract: Humans perform complex tasks involving force interactions daily. Learning from demonstration, a method for transferring such human manipulation skills to robots, requires techniques for segmenting the demonstrations into movement primitives. Therefore, we propose an unsupervised motion segmentation method that utilizes small characteristic fluctuations of 6-axis force/torque signals as features for motion segmentation. This method includes a feature extraction phase using a differentiation process and detects segmentation points based on the differentiated 6-axis force/torque signals obtained in the task demonstrations. The segmentation method was evaluated using a peg-in-hole task and bottle-lid opening task. The experimental results demonstrate the validity of using differentiated forces and torques for motion segmentation.

Keywords: Learning from Demonstration , Physical Human-Robot Interaction , Compliance and Impedance Control
Abstract: Skill propagation among robots without human involvement can be crucial in quickly spreading new physical skills to many robots. In this respect, it is a good alternative to pure reinforcement learning, which can be time-consuming, or learning from human demonstration, which requires human involvement. In the latter case, there may not be enough humans to quickly spread skills to many robots. However, propagation among robots without direct human supervision can result in robotic skills mutating from the original source. This can be beneficial when better skills might emerge or when a new skill is obtained to be used for other similar tasks. However, it can also be dangerous in terms of task execution safety. This letter studies the mutation of a robotic skill when it is propagated from one robot to another during a physically collaborative task. We chose the collaborative sawing task as a study case since it involves complex two-agent physical interaction/coordination and because its periodic nature can facilitate repetitive learning. The study employs periodic Dynamic Movement Primitives and Locally Weight Regression to encode and learn the motion and impedance required to execute the task. To explore what influences mutation, we varied several control and environment conditions such as the maximum stiffness, robot base position, friction coefficient of the sawed object, and movement period.

Keywords: Learning from Demonstration , Representation Learning , Force and Tactile Sensing
Abstract: In many contact-rich tasks, force sensing plays an essential role in adapting the motion to the physical properties of the manipulated object. To enable robots to capture the underlying distribution of object properties necessary for generalising learnt manipulation tasks to unseen objects, existing Learning from Demonstration (LfD) approaches require a large number of costly human demonstrations. Our proposed semi-supervised LfD approach decouples the learnt model into a haptic representation encoder and a motion generation decoder. This enables us to pre-train the first using a large amount of unsupervised data, easily accessible, while using few-shot LfD to train the second, leveraging the benefits of learning skills from humans. We validate the approach on the wiping task using sponges with different stiffness and surface friction. Our results demonstrate that pre-training significantly improves the ability of the LfD model to recognise physical properties and generate desired wiping motions for unseen sponges, outperforming the LfD method without pre-training. We validate the motion generated by our semi-supervised LfD model on the physical robot hardware using the KUKA iiwa robot arm. We also validate that the haptic representation encoder, pre-trained in simulation, captures the properties of real objects, explaining its contribution to improving the generalisation of the downstream task.

Keywords: Learning from Demonstration , Planning under Uncertainty , Simulation and Animation
Abstract: Building a general and efficient path planning framework in uncertain nonconvex environments is challenging due to the safety constraints and complex configuration. Traditional avenues usually involve convexifying obstacles and presume Gaussian distribution, which are not universal. Meanwhile, the fast convergence of high-quality solutions is not guaranteed. Therefore, we develop a novel neural risk-bounded path planner to quickly find near-optimal solutions that have an acceptable collision probability in the complex environments. Firstly, we retrieve the nonconvex obstacles with arbitrary probabilistic uncertainties in the form of a deterministic point cloud map. A neural network sampler encodes it into a latent embedding and is trained with sufficient expert demonstrations, predicting states in the potential subspace. We construct a neural cost estimator to select the best informed state from those samples. Then, we recursively use the simple yet effective neural networks to march toward the start and goal bidirectionally. The collision risk of the intermediate connections is verified based on sum-of-squares optimization. Simulation results show that our approach significantly saves time and resources in finding comparable solutions over the state-of-the-art methods in the seen and unseen challenging environments.

Keywords: Learning from Demonstration , Sensorimotor Learning , Machine Learning for Robot Control
Abstract: Large-scale self-supervised models have recently revolutionized our ability to perform a variety of tasks within the vision and language domains. However, using such models for autonomous systems is challenging because of safety requirements: besides executing correct actions, an autonomous agent must also avoid the high cost and potentially fatal critical mistakes. Traditionally, self-supervised training mainly focuses on imitating previously observed behaviors, and the training demonstrations carry no notion of which behaviors should be explicitly avoided. In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion. ConBaT is inspired by the concept of control barrier functions in control theory and uses a causal transformer that learns to predict safe robot actions autoregressively using a critic that requires minimal safety data labeling. During deployment, we employ a lightweight online optimization to find actions that ensure future states lie within the learned safe set. We apply our approach to different simulated control tasks and show that our method results in safer control policies compared to other classical and learning-based methods such as imitation learning, reinforcement learning, and model predictive control.

Keywords: Learning from Demonstration , Task and Motion Planning
Abstract: As robots tackle complex object arrangement tasks, it becomes imperative for them to be able to generalize to complex worlds and scale with number of objects. This work postulates that extracting action primitives, such as push operations, their pre-conditions and effects would enable strong generalization to unseen worlds. Hence, we factorize policy learning as inference of such generic rules, which act as strong priors for predicting actions given the world state. Learnt rules act as propositional knowledge and enable robots to reach goals in a zero-shot method by applying the rules independently and incrementally. However, obtaining hand-engineered rules, such as PDDL descriptions is hard, especially for unseen worlds. This work aims to learn generic, sparse, and context-aware rules that govern action primitives in robotic worlds through human demonstrations in simple domains. We demonstrate that our approach, namely RLAP, is able to extract rules without explicit supervision of rule labels and generate goal-reaching plans in complex Sokoban styled domains that scale with number of objects. RLAP furnishes significantly higher goal reaching rate and shorter planning times compared to the state-of-the-art techniques. The code, dataset, and videos are hosted at https://rule-learning-rlap.github.io/.

Keywords: Deep Learning Methods , Deep Learning for Visual Perception
Abstract: The ability to predict future structure features of environments based on past perception information is extremely needed by autonomous vehicles, which helps to make the following decision-making and path planning more reasonable. Recently, point cloud prediction (PCP) is utilized to predict and describe future environmental structures by the point cloud form. In this letter, we propose a novel efficient Transformer-based network to predict the future LiDAR point clouds exploiting the past point cloud sequences. We also design a semantic auxiliary training strategy to make the predicted LiDAR point cloud sequence semantically similar to the ground truth and thus improves the significance of the deployment for more tasks in real-vehicle applications. Our approach is completely self-supervised, which means it does not require any manual labeling and has a solid generalization ability toward different environments. The experimental results show that our method outperforms the state-of-the-art PCP methods on the prediction results and semantic similarity, and has a good real-time performance. Our open-source code and pre-trained models are available at https://github.com/Blurryface0814/PCPNet.

Keywords: Learning and Adaptive Systems , Mobile Manipulation , Service Robots , Robot Learning
Abstract: Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons. Ex- isting methods struggle to control the large configuration space and to navigate dynamic and unknown environments. In the previous work, we proposed to decompose mobile manipulation tasks into a simplified motion generator for the end-effector in task space and a trained reinforcement learning agent for the mobile base to account for the kinematic feasibility of the motion. In this work, we introduce Neural Navigation for Mobile Manipulation (N2M2), which extends this decomposition to complex obstacle environ- ments, extends the agent’s control to the torso joint and the norm of the end-effector motion velocities, uses a more general reward function and, thereby, enables robots to tackle a much broader range of tasks in real-world settings. The resulting approach can perform unseen, long-horizon tasks in unexplored environments while instantly reacting to dynamic obstacles and environmental changes. At the same time, it provides a simple way to define new mobile manipulation tasks. We demonstrate the capabilities of our proposed approach in extensive simulation and real-world experi- ments on multiple kinematically diverse mobile manipulators.

Keywords: Learning and Adaptive Systems , Reactive and Sensor-Based Planning , Deep Learning in Robotics and Automation , Indoor Navigation
Abstract: We present a method for generating, predicting, and using Spatiotemporal Occupancy Grid Maps (SOGM), which embed future semantic information of real dynamic scenes. We present an auto-labeling process that creates SOGMs from noisy real navigation data. We use a 3D-2D feedforward architecture, trained to predict the future time steps of SOGMs, given 3D lidar frames as input. Our pipeline is entirely self-supervised, thus enabling lifelong learning for real robots. The network is composed of a 3D back-end that extracts rich features and enables the semantic segmentation of the lidar frames, and a 2D front-end that predicts the future information embedded in the SOGM representation, potentially capturing the complexities and uncertainties of real-world multi-agent interactions. We also design a navigation system that uses these predicted SOGMs within planning, after they have been transformed into Spatiotemporal Risk Maps (SRMs). We verify our navigation system's abilities in simulation, validate it on a real robot, study SOGM predictions on real data in various circumstances, and provide a novel indoor 3D lidar dataset, collected during our experiments, which includes our automated annotations.

Keywords: Deep Learning Methods , Imitation Learning , Representation Learning
Abstract: In policy learning for robotic manipulation, sample efficiency is of paramount importance. Thus, learning and extracting more compact representations from camera observations is a promising avenue. However, current methods often assume full observability of the scene and struggle with scale invariance. In many tasks and settings, this assumption does not hold as objects in the scene are often occluded or lie outside the field of view of the camera, rendering the camera observation ambiguous with regard to their location. To tackle this problem, we present BASK, a Bayesian approach to tracking scale-invariant keypoints over time. Our approach can successfully resolve inherent ambiguities in images, enabling keypoint tracking on symmetrical objects and occluded and out-of-view objects. We employ ourmethod to learn challenging multi-object robot manipulation tasks from wrist camera observations and demonstrate superior utility for policy learning compared to other representation learning techniques. Furthermore, we show outstanding robustness towards disturbances such as clutter, occlusions, and noisy depth measurements, as well as generalization to unseen objects both in simulation and real-world robotic experiments.

Keywords: Deep Learning Methods , AI-Based Methods , Intelligent Transportation Systems
Abstract: Trajectory prediction plays a crucial role in the autonomous driving stack by enabling autonomous vehicles to anticipate the motion of surrounding agents. Goal-based prediction models have gained traction in recent years for addressing the multimodal nature of future trajectories. Goal-based prediction models simplify multimodal prediction by first predicting 2D goal locations of agents and then predicting trajectories conditioned on each goal. However, a single 2D goal location serves as a weak inductive bias for predicting the whole trajectory, often leading to poor map compliance, i.e., part of the trajectory going off-road or breaking traffic rules. In this paper, we improve upon goal-based prediction by proposing the Path-based prediction (PBP) approach. PBP predicts a discrete probability distribution over reference paths in the HD map using the path features and predicts trajectories in the path-relative Frenet frame. We applied the PBP trajectory decoder on top of the HiVT scene encoder and report results on the Argoverse dataset. Our experiments show that PBP achieves competitive performance on the standard trajectory prediction metrics, while significantly outperforming state-of-the-art baselines in terms of map compliance.

Keywords: Deep Learning Methods , Physical Human-Robot Interaction , Industrial Robots
Abstract: Physical human-robot interaction has been an area of interest for decades. Collaborative tasks, such as joint compliance, demand high-quality joint torque sensing. While external torque sensors are reliable, they come with the drawbacks of being expensive and vulnerable to impacts. To address these issues, studies have been conducted to estimate external torques using only internal signals, such as joint states and current measurements. However, insufficient attention has been given to friction hysteresis approximation, which is crucial for tasks involving extensive dynamic to static state transitions. In this paper, we propose a deep-learning-based method that leverages a novel long-term memory scheme to achieve dynamics identification, accurately approximating the static hysteresis. We also introduce modifications to the well-known Residual Learning architecture, retaining high accuracy while reducing inference time. The robustness of the proposed method is illustrated through a joint compliance and task compliance experiment.

Keywords: Deep Learning Methods , Transfer Learning , Reinforcement Learning
Abstract: Recent work has shown the promise of creating generalist, transformer-based, models for language, vision, and sequential decision-making problems. To create such models, we generally require centralized training objectives, data, and compute. It is of interest if we can more flexibly create generalist policies by merging together multiple, task-specific, individually trained policies. In this work, we take a preliminary step in this direction through merging, or averaging, subsets of Decision Transformers in parameter space trained on different MuJoCo locomotion problems, forming multi-task models without centralized training. We also demonstrate the importance of various methodological choices when merging policies, such as utilizing common pre-trained initializations, increasing model capacity, and utilizing Fisher information for weighting parameter importance. In general, we believe research in this direction could help democratize and distribute the process that forms multi-task robotics policies. Our implementation is available at https://github.com/daniellawson9999/merging-decision-transf ormers.

Keywords: Human-Robot Collaboration , Formal Methods in Robotics and Automation , Task Planning
Abstract: Recent work has considered trust-aware decision making for human-robot collaboration (HRC) with a focus on model learning. In this paper, we are interested in enabling the HRC system to complete complex tasks specified using temporal logic formulas that involve human trust. Since accurately observing human trust in robots with precision is challenging, we adopt the widely used partially observable Markov decision process (POMDP) framework for modelling the interactions between humans and robots. To specify the desired behaviour, we propose to use syntactically co-safe linear distribution temporal logic (scLDTL), a logic that is defined over predicates of states as well as belief states of partially observable systems. The incorporation of belief predicates in scLDTL enhances its expressiveness while simultaneously introducing added complexity. This also presents a new challenge as the belief predicates must be evaluated over the continuous (infinite) belief space. To address this challenge, we present an algorithm for solving the optimal policy synthesis problem. First, we enhance the belief MDP (derived by reformulating the POMDP) with a probabilistic labelling function. Then a product belief MDP is constructed between the probabilistically labelled belief MDP and the automaton translation of the scLDTL formula. Finally, we show that the optimal policy can be obtained by leveraging existing point-based value iteration algorithms with essential modifications. Human subject experiments with 21 participants on a driving simulator demonstrate the effectiveness of the proposed approach.

Keywords: Human-Robot Collaboration , Human-Robot Teaming , Safety in HRI
Abstract: We focus on the problem of how we can enable a robot to collaborate seamlessly with a human partner, specifically in scenarios where preexisting data is sparse. Much prior work in human-robot collaboration uses observational models of humans (i.e. models that treat the robot purely as an observer) to choose the robot's behavior, but such models do not account for the influence the robot has on the human's actions, which may lead to inefficient interactions. We instead formulate the problem of optimally choosing a collaborative robot's behavior based on a conditional model of the human that depends on the robot's future behavior. First, we propose a novel model-based formulation of conditional behavior prediction that allows the robot to infer the human's intentions based on its future plan in data-sparse environments. We then show how to utilize a conditional model for proactive goal selection and safe trajectory generation around human collaborators. Finally, we use our proposed proactive controller in a collaborative task with real users to show that it can improve users' interactions with a robot collaborator quantitatively and qualitatively.

Keywords: Human-Robot Collaboration , Human-Robot Teaming , Social HRI
Abstract: Despite advancements in human-robot teamwork, limited progress was made in developing AI assistants capable of advising teams online during task time, due to the challenges of modeling both individual and collective beliefs of the team members. Dynamic epistemic logic has proved to be a viable tool for representing a machine Theory of Mind and for modeling communication in epistemic planning, with applications to human-robot teamwork. However, this approach has yet to be applied in an online teaming assistance context and fails to account for the real-life probabilities of potential team beliefs. We propose a novel blend of epistemic planning and POMDP techniques to create a risk-bounded AI team assistant, that intervenes only when the team's expected likelihood of failure exceeds a predefined risk threshold or in the case of potential execution deadlocks. Our experiments and simulated demonstration on the Virtualhome testbed show that the assistant can effectively improve team performance.

Keywords: Human-Robot Collaboration , Compliant Assembly , Human-Centered Automation
Abstract: This paper addresses human-robot collaborative (HRC) precision assembly that complements natural human ability and the strength of an autonomous robot system. Our approach enables both flexibility and efficiency of tight-clearance assembly of various complex-shaped parts in the presence of uncertainty without requiring assembly skills and knowledge of robotics from the human operator. We demonstrated the effectiveness of our approach in a variety of experiments and comparisons with other HRC assembly approaches.

Keywords: Human-Robot Collaboration , Human Factors and Human-in-the-Loop , SLAM
Abstract: Simultaneous Localization and Mapping (SLAM) has emerged as a prime autonomous mobile agent localization algorithm. Despite the global research effort to improve SLAM, its mapping component remains limited and serves little more than to satisfy the coupled localization problem. We present a collaborative 3D SLAM approach leveraging the power of augmented reality (AR). The system introduces a trio of diverse agents, each with its unique capability to become an active member in the mapping process: mobile robots, human operators, and AR head-mounted display (AR-HMD). A 3D complementary mapping pipeline is developed to utilize the built-in SLAM capabilities of the AR-HMD as shareable data. Our system aligns and merges the AR-HMD and the robot’s local map automatically, triggered by a human-dictated initial guess. The created merged map proves advantageous in scenarios where the robot is restricted from navigating in certain areas. To correct map imperfections resulting from problematic objects such as transparent or reflective surfaces, the fused map is overlayed onto the environment, and hand gestures are used to add or delete 3D map features in real-time. Our system is implemented in both a lab and a real industrial warehouse setup. The results show a significant improvement in the map quality and mapping duration.

Keywords: Human-Robot Collaboration , Deep Learning Methods , Machine Learning for Robot Control
Abstract: The accumulation of a sufficient amount of data for training deep neural networks is a major hindrance in the application of deep learning in robotics. Acquiring real-world data requires considerable time and effort, yet it might still not capture the full range of potential environmental variations. The generation of new synthetic data based on existing training data has been enabled with the development of generative adversarial networks (GANs). In this paper, we introduce a training methodology based on GANs that utilizes a recurrent, LSTM-based architecture for intention recognition in robotics. The resulting networks predict the intention of the observed human or robot based on input RGB videos. They are trained in a semi-supervised manner, with the output classification networks predicting one of possible labels for the observed motion, while the recurrent generator networks produce fake RGB videos that are leveraged in the training process. We show that utilization of the generated data during the network training process increases the accuracy and generality of motion classification compared to using only real training data. The proposed method can be applied to a variety of dynamic tasks and different LSTM-based classification networks to supplement real data.

Keywords: Human-Robot Collaboration , Human-Centered Automation , Learning from Demonstration
Abstract: Mass customization and shorter manufacturing cycles are becoming more important among small and medium-sized companies. However, classical industrial robots struggle to cope with product variation and dynamic environments. In this paper, we present CoBT, a collaborative programming by demonstration framework for generating reactive and modular behavior trees. CoBT relies on a single demonstration and a combination of data-driven machine learning methods with logic-based declarative learning to learn a task, thus eliminating the need for programming expertise or long development times. The proposed framework is experimentally validated on 7 manipulation tasks and we show that CoBT achieves approx 93% success rate overall with an average of 7.5s programming time. We conduct a pilot study with non-expert users to provide feedback regarding the usability of CoBT. More videos and generated behavior trees are available at:https://github.com/jainaayush2006/CoBT.git.

Keywords: Human-Robot Collaboration , Human-Aware Motion Planning , Safety in HRI
Abstract: The new face of modern industrial scenarios involves shared workspaces where humans and robots work closely together. To ensure safe human-robot collaboration (HRC), regulations have been updated introducing the ISO/TS15066. However, complying with these regulations often leads to inefficient behavior, such as unnecessarily reducing robot speed or unpredictably changing the robot path, which may negatively affect the operator perception of the robot. In this work an optimal approach to address together these two issues is proposed. Starting from a desired final configuration, the framework plans a collision-free trajectory for the robot. Subsequently, predictability is taken into account and a set of virtual tubes into which the path of the robot can move is built. Lastly, an optimization problem is solved online to ensure that the robot stays within these tubes and the velocities are compliant with the ISO/TS 15066. The proposed approach has been experimentally validated in two different scenarios: one composed by a mobile manipulator, i.e. a UR10e mounted on a Neobotix MPO-500, and one composed by only a collaborative manipulator, i.e. a UR5e.

Keywords: Vision-Based Navigation , Representation Learning , Autonomous Vehicle Navigation
Abstract: Autonomous mobility tasks such as last-mile delivery require reasoning about operator-indicated preferences over terrains on which the robot should navigate to ensure both robot safety and mission success. However, coping with out-of-distribution data from novel terrains or appearance changes due to lighting variations remains a fundamental problem in visual terrain-adaptive navigation. Existing solutions either require labor-intensive manual data re-collection and labeling or use hand-coded reward functions that may not align with operator preferences. In this work, we posit that operator preferences for visually novel terrains, which the robot should adhere to, can often be extrapolated from established terrain preferences within the inertial-proprioceptive-tactile domain. Leveraging this insight, we introduce Preference extrApolation for Terrain-awarE Robot Navigation (PATERN), a novel framework for extrapolating operator terrain preferences for visual navigation. PATERN learns to map inertial-proprioceptive-tactile measurements from the robot’s observations to a representation space and performs a nearest-neighbor search in this space to estimate operator preferences over novel terrains. Through physical robot experiments in outdoor environments, we assess PATERN’s capability to extrapolate preferences and generalize to novel terrains and challenging lighting conditions. Compared to baseline approaches, our findings indicate that PATERN robustly generalizes to diverse terrains and varied lighting conditions, while navigating in a preference-aligned manner.

Keywords: Human and Humanoid Motion Analysis and Synthesis , Natural Machine Motion , Bimanual Manipulation
Abstract: Generating human-like robot motions is pivotal for achieving smooth human-robot interactions. Such motions contribute to better predictions of robot motions by humans, thus leading to more intuitive interaction and increased acceptability. Human likeness in robot motions has been conventionally measured and realized via the optimization of human-likeness metrics. However, the abundance of such metrics and the absence of standardized criteria impede their usage in novel contexts. In this work, we introduce a unified human-likeness metric built from a hierarchically weighted sum of individual metrics. The proposed metric is derived from a thorough analysis of eleven existing human-likeness criteria and is applicable across various tasks and robot models. We evaluate its performance in the context of motion retargeting of bimanual tasks with three different humanoid robots.

Keywords: Health Care Management , Medical Robots and Systems , Rehabilitation Robotics
Abstract: Massage therapy is helpful for the rehabilitation of various diseases, such as headaches caused by migraines and stress. Existing robotic systems have focused on massage therapy on the torso and limbs, but performing massage motions through suitable actuation on a person’s head has been a challenge. In this paper, we present MiBOT, a head-worn massage robot that actuates two soft tactors to produce touch motions mimicking human massage. A key design principle behind MiBOT is its silent actuation, which we achieve through pneumatic artificial muscles in conjunction with a controller loop to respond to contact pressure. We evaluated the effectiveness of MiBOT in a controlled study and assessed subjects’ blood pressure and heart rate levels while applying MiBOT. We found that our mechanical system generated positive and conclusive quantitative outcomes that are similar to the human-administered massage, decreasing participants’ mean systolic and diastolic blood pressure by 2.8 mmHg and 1.7 mmHg, respectively, as well as calming their heart rate by 8–10% on average.

Keywords: Intelligent Transportation Systems , Datasets for Human Motion , Data Sets for Robot Learning
Abstract: Emergent-scene safety is the key milestone for fully autonomous driving, and reliable on-time prediction is essential to maintain safety in emergency scenarios. However, these emergency scenarios are long-tailed and hard to collect, which restricts the system from getting reliable predictions. In this paper, we build a new dataset, which aims at the long-term prediction with the inconspicuous state variation in history for the emergency event, named the Extro-Spective Prediction (ESP) problem. Based on the proposed dataset, a flexible feature encoder for ESP is introduced to various prediction methods as a seamless plug-in, and its consistent performance improvement underscores its efficacy. Furthermore, a new metric named clamped temporal error (CTE) is proposed to give a more comprehensive evaluation of prediction performance, especially in time-sensitive emergency events of subseconds. Interestingly, as our ESP features can be described in human-readable language naturally, the application of integrating into ChatGPT also shows huge potential. The ESP-dataset and all benchmarks are released at url{https://dingrui-wang.github.io/ESP-Dataset/

Keywords: Modeling and Simulating Humans , Datasets for Human Motion , Human and Humanoid Motion Analysis and Synthesis
Abstract: Synthetic data generation is a proven method for augmenting training sets without the need for extensive setups, yet its application in human activity recognition is underexplored. This is particularly crucial for human-robot collaboration in household settings, where data collection is often privacy-sensitive. In this paper, we introduce SynthAct, a synthetic data generation pipeline designed to significantly minimize the reliance on real-world data. Leveraging modern 3D pose estimation techniques, SynthAct can be applied to arbitrary 2D or 3D video action recordings, making it applicable for uncontrolled in-the-field recordings by robotic agents or smarthome monitoring systems. We present two SynthAct datasets: AMARV, a large synthetic collection with over 800k multi-view action clips, and Synthetic Smarthome, mirroring the Toyota Smarthome dataset. SynthAct generates a rich set of data, including RGB videos and depth maps from four synchronized views, 3D body poses, normal maps, segmentation masks and bounding boxes. We validate the efficacy of our datasets through extensive synthetic-to-real experiments on NTU RGB+D and Toyota Smarthome.

Keywords: Modeling and Simulating Humans , Deep Learning in Grasping and Manipulation , Force and Tactile Sensing
Abstract: The soft fingers and strategic grasping skills enable the human hands to grasp objects stably. This paper aims to model human grasping skills and transfer the learned skills to robots to improve grasping quality and success rate. First, we designed a wearable tool-like parallel hand exoskeleton equipped with optical tactile sensors to acquire multimodal information, including hand positions and postures, the relative distance of the exoskeleton claws, and tactile images. Using the demonstration data, we summarized three characteristics observed from human demonstrations: varying-speed actions, grasping effect read from tactile images and grasping strategies for different positions. The characteristics were then utilized in the robot skill modelling to achieve a more human-like grasp. Since no force sensors are fixed to the claws, we introduced a new variable, called "grasp depth", to represent the grasping effect on the object. The robot grasping strategy diagram is constructed as follows: First, grasp quality is predicted using a linear array network (LAN) and global visual images as inputs. The conditions such as grasp width, depth, position, and angle are also predicted. Second, with the grasp width and depth of the object determined, dynamic movement primitives (DMPs) are employed to mimic human grasp actions with varying velocities. A final action adjustment based on tactile detection is performed during the near-grasp time to enhance grasp quality further.

Keywords: Modeling and Simulating Humans , Human and Humanoid Motion Analysis and Synthesis
Abstract: In industrial workstations, the morphology of the worker is a key factor for the feasibility and the ergonomics of an activity. Existing digital human modeling tools can simulate different morphologies at work, but hardly scale to a large population of workers because of limited consideration of morphology-specific behaviors and computational cost. This paper presents a framework to efficiently evaluate the suitability of a workstation over a large population of workers in a physics-based simulation. Activities are simulated through a two-step optimization process, involving a quadratic-programming based whole-body controller and a multi-task optimizer for behavioral adaptation. On a screwdriving scenario, we demonstrate how our framework can help ergonomists improve workstation designs thanks to the resulting suitability maps where generated behaviors are optimized for each morphology w.r.t. ergonomics and performance.

Keywords: Modeling and Simulating Humans , Human Factors and Human-in-the-Loop , Humanoid and Bipedal Locomotion
Abstract: Modeling and control of the human musculoskeletal system is important for understanding human motor functions, developing embodied intelligence, and optimizing human-robot interaction systems. However, current open-source models are restricted to a limited range of body parts and often with a reduced number of muscles. There is also a lack of algorithms capable of controlling over 600 muscles to generate reasonable human movements. To fill this gap, we build a musculoskeletal model with 90 body segments, 206 joints, and 700 muscle-tendon units, allowing simulation of full-body dynamics and interaction with various devices. We develop a new algorithm using low-dimensional representation and hierarchical deep reinforcement learning to achieve state-of-the-art full-body control. We validate the effectiveness of our model and algorithm in simulations with real human locomotion data. The musculoskeletal model, along with its control algorithm, will be made available to the research community to promote a deeper understanding of human motion control and better design of interactive robots.

Keywords: Modeling and Simulating Humans , Human-Centered Automation , Deep Learning for Visual Perception
Abstract: Single-view 3D human reconstruction has been a hot topic due to the potential of wide applications. To achieve high accuracy, existing works usually take computationally intensive models as backbone for exhaustive underlying features and then directly estimate human mesh vertices. These factors lead to redundant parameters, large calculations and low efficiency, while lightweight solutions to address these challenges are relatively scarce. In this work, based on the problems studied above, we propose a keypoints-guided lightweight network with an encoding-decoding framework. As the input is an image, a lightweight backbone named multi-stage and global feature enhanced network is designed for 2D encoding, where some operations of multi-scale fusion and frequency domain filtering are performed to extract more informative but low-resolution features. As the output is mesh of human body, we construct a keypoints-based 3D human template, with which the 2D low-resolution features can be mapped to 3D space to guide the 3D decoding with high efficiency and high accuracy. Extensive experiments on popular benchmarks 3DPW and Human3.6M illustrate the favorable trade-off between the accuracy and complexity of our method. Our code is publicly available at https://github.com/ChrisChenYh/EfficientHuman.git.

Keywords: Modeling and Simulating Humans , Physical Human-Robot Interaction , Sensor-based Control
Abstract: Human-robot interaction based on real-time kinematics or electromyography (EMG) feedback improves rehabilitation using assist-as-needed strategies. Muscle forces are expected to provide even more comprehensive information than EMG to control these assistive rehabilitation devices.

Measuring in vivo muscle force is challenging, leading to the development of numerical methods to estimate them. Due to their high computational cost, forward dynamics-based optimization algorithms were not viable for real-time estimation until recently. To achieve muscle forces estimation in real time, a moving horizon estimator (MHE) algorithm was used to track experimental biosignals. Two participants were equipped with EMG sensors and skin markers that were streamed in real time and used as targets for the MHE. The upper-limb musculoskeletal (MSK) model was composed of 10 degrees-of-freedom actuated by 31 muscles. The MHE relies on a series of overlapping trajectory optimization subproblems of which the following parameters have been adjusted: the fixed duration and the frame to export. We based this adjustment on the estimation delay, the muscle saturation, the joint kinematic mean power frequency, and errors to experimental data. Our algorithm provided consistent estimates of muscle forces and kinematics with visual feedback at 30 Hz with a 110 ms delay. This method is promising to guide rehabilitation and enrich assistive device control laws with personalized force estimations.

Keywords: Humanoid and Bipedal Locomotion , Model Learning for Control , Reinforcement Learning
Abstract: This paper presents a probabilistic Model-based Reinforcement Learning (MBRL) approach for learning the Energy-exchange Dynamics (EED) of a spring-loaded biped robot. Our approach enables on-site walking acquisition with high sample efficiency, real-time planning capability and generalizability across skill conditions. Specifically, we learn the data-driven state transition dynamics of the robot in the formulation of energy-states, with their interaction characterized as energy-exchange to reduce dimensionality. To improve planning reliability with the learned EED, we design a control space based on a walking trajectory that follows the law of conservation of energy and formulated by energy-states. We evaluated our approach using a four-degree-of-freedom spring-loaded biped robot in simulation and hardware, and generalizability is validated by using same learning framework for different walking speeds and terrains in simulation and walking acquisition with hardware. All results showed successful on-site walking acquisition with a compact nine-dimension dynamics model, 40Hz real-time planning, and on-site learning within a few minutes.}

Keywords: Humanoid and Bipedal Locomotion , Modeling and Simulating Humans
Abstract: Legged robots have demonstrated high efficiency and effectiveness in unstructured and dynamic environments. However, it is still challenging for legged robots to achieve rapid and efficient locomotion on deformable, yielding substrates, such as granular terrains. We present an enhanced resistive force model for bipedal walkers on soft granular terrains by introducing effective intrusion depth correction. The enhanced force model captures fundamental kinetic results considering the robot foot shape, walking gait speed variation, and energy expense. The model is validated by extensive foot intrusion experiments with a bipedal robot. The results confirm the model accuracy on the given type of granular terrains. The model can be further integrated with the motion control of bipedal robotic walkers.

Keywords: Humanoid and Bipedal Locomotion , Passive Walking , Robust/Adaptive Control
Abstract: In this paper, we propose an adaptive controller for virtual passive biped dynamic walking on unknown uneven terrain. The adaptive controller consists of a trajectory tracking control law developed via backstepping method to mimic reference passive gait, and a slope estimator for the inclination angle of the terrain. In addition, a re-planning approach is introduced to correct the robot state off-track from the reference gait due to the terrain changes. The controller is validated through the simulations on the mixed uneven terrain consisting of varying slopes and steps. The results suggest that the controller shows comparable cost of transport and greater adaptability to terrain changes compared with certain existing methods.

Keywords: Humanoid and Bipedal Locomotion , Reinforcement Learning , Human and Humanoid Motion Analysis and Synthesis
Abstract: Transferring human motion skills to humanoid robots remains a significant challenge. In this study, we introduce a Wasserstein adversarial imitation learning system, allowing humanoid robots to replicate natural whole-body locomotion patterns and execute seamless transitions by mimicking human motions. First, we present a unified primitive-skeleton motion retargeting to mitigate morphological differences between arbitrary human demonstrators and humanoid robots. An adversarial critic component is integrated with Reinforcement Learning (RL) to guide the control policy to produce behaviors aligned with the data distribution of mixed reference motions. Additionally, we employ a specific Integral Probabilistic Metric (IPM), namely the Wasserstein-1 distance with a novel soft boundary constraint to stabilize the training process and prevent model collapse. Our system is evaluated on a full-sized humanoid JAXON in the simulator. The resulting control policy demonstrates a wide range of locomotion patterns, including standing, push-recovery, squat walking, human-like straight-leg walking, and dynamic running. Notably, even in the absence of transition motions in the demonstration dataset, the robot showcases an emerging ability to transit naturally between distinct locomotion patterns as desired speed changes.

Keywords: Humanoid and Bipedal Locomotion , Whole-Body Motion Planning and Control
Abstract: For humanoid robots, online motion generation on non-flat terrain remains an ongoing research challenge. Computational complexity is one of the primary restrictions that preclude motion planners from generating adaptive behaviors online. In this paper, we investigate this problem and decompose it into two sequential components: an Efficient Behavior Generator (EBG) and a Nonlinear Centroidal Model Predictive Controller (NC-MPC). The EBG is responsible for optimizing the physically feasible whole-body template behaviors, which can provide reliable warm-starts for NC-MPC, thereby greatly reducing the computational effort of online planning. With tailored objective function and feet complementary constraints, the EBG can search for a near-optimal solution after several iterations within seconds for different behaviors including walking, running, and jumping, even with intuitive initial guesses. To make the template behaviors extensible when the robot encounters possible different scenarios, the NC-MPC is proposed to regenerate the reactive motion online to adapt it to the real local environment. Finally, we validate the effectiveness of synthesizing EBG and NC-MPC for humanoid locomotion on non-flat terrain in simulation and on the real humanoid robot BHR7P.

Keywords: Humanoid Robot Systems , Biomimetics
Abstract: In this letter, we present a method for realizing the heat-evoked nociceptive withdrawal reflex (NWR) in the upper limb of a humanoid robot so that it can avoid the potential damage caused by noxious heat.

We use a spiking neuron network whose structure, encoding scheme, and form of information transmission mimic the reflex arc in humans to improve bio-plausibility. The proper synaptic strengths between the sensory neurons and the interneurons in the first two layers are learned using the bio-plausible reward modulated spike timing-dependent

plasticity learning algorithm. By monitoring the spikes from the motor neuron in the third layer, a reflex matching the intensity of the stimulation can be evoked. Experimental evaluations show that noxious heat stimulation can be detected online and evoke the NWR. The experiments on a full-size humanoid robot show that the method enables robots to avoid potential damage robustly with proper NWR, depending on

the site and intensity of the stimulation. We also verify that the method takes advantage of the intrinsic characteristics of its neuromorphic encoding scheme to reproduce essential features of the NWR

e.g., spatial summation effect and temporal summation effect in humans.

The improved bio-plausibility and the capability to reproduce the human-like features make the proposed method suitable for devices that provide perceptual feedback to human users and allow local processing with low energy consumption, such as cognitive prosth

Keywords: Humanoid Robot Systems , Failure Detection and Recovery , Humanoid and Bipedal Locomotion
Abstract: This paper presents a novel approach to fall prediction for bipedal robots, specifically targeting the detection of potential falls while standing caused by abrupt, incipient, and intermittent faults. Leveraging a 1D convolutional neural network (CNN), our method aims to maximize lead time for fall prediction while minimizing false positive rates. The proposed algorithm uniquely integrates the detection of various fault types and estimates the lead time for potential falls. Our contributions include the development of an algorithm capable of detecting abrupt, incipient, and intermittent faults in full-sized robots, its implementation using both simulation and hardware data for a humanoid robot, and a method for estimating lead time. Evaluation metrics, including false positive rate, lead time, and response time, demonstrate the efficacy of our approach. Particularly, our model achieves impressive lead times and response times across different fault scenarios with a false positive rate of 0. The findings of this study hold significant implications for enhancing the safety and reliability of bipedal robotic systems.

Keywords: Humanoid Robot Systems , Modeling, Control, and Learning for Soft Robots
Abstract: Shape-changing robotic mannequin is a humanoid robot for imitating shapes of human bodies. The diversity of human bodies makes it difficult to imitate various body shapes, especially the shoulders. This paper proposes a rigid-flexible-soft coupling three-layered robotic mannequin shoulder inspired by human body anatomy. The robotic mannequin can adjust the anisotropic deformation of its human-like skin to imitate body dimensions, shape details and surface curvatures of target bodies. Structurally, the inner skeleton layer is composed of rigid framework and linear actuators for changing the global body dimensions. The middle muscle layer consists of flexible patches and layer-jamming bars with tunable stiffness for controlling the surface curvatures. The outer soft skin layer envelops the patches, forming a human-like surface of the robotic mannequin. To imitate a human body, the linear actuators drive the patches forward, which deforms the elastic skin layer. The tensioned skin layer inversely drives the bending deformation of patches, which can be controlled by layer-jamming bars. We design the three-layered structure by analyzing the shape differences of hundreds of scanned human models. An energy-based method is proposed to predict and control the coupling deformation of the layered structure. A physical robotic shoulder prototype has been built to verify the effectiveness of our method.

Keywords: Humanoid Robot Systems , Physical Human-Robot Interaction , Sensor Fusion
Abstract: This paper proposes a novel sensor fusion based on Unscented Kalman Filtering for the online estimation of joint-torques of humanoid robots without joint-torque sensors. At the feature level, the proposed approach considers multi-modal measurements (e.g. currents, accelerations, etc.) and non-directly measurable effects, such as external contacts, thus leading to joint torques readily usable in control architectures for human-robot interaction. The proposed sensor fusion can also integrate distributed, non-collocated force/torque sensors, thus being a flexible framework with respect to the underlying robot sensor suit. To validate the approach, we show how the proposed sensor fusion can be integrated into a two-level torque control architecture aiming at task-space torque control. The performances of the proposed approach are shown through extensive tests on the new humanoid robot ergoCub, currently being developed at Istituto Italiano di Tecnologia. We also compare our strategy with the existing state-of-the-art approach based on the recursive Newton-Euler algorithm. Results demonstrate that our method achieves low root mean square errors in torque tracking, ranging from 0.05 Nm to 2.5 Nm, even in the presence of external contacts.

Keywords: Reactive and Sensor-Based Planning , Collision Avoidance , Aerial Systems: Applications
Abstract: Fast and reliable obstacle avoidance is an important task for mobile robots. In this work, we propose an efficient reactive system that provides high-quality obstacle avoidance while running at hundreds of hertz with minimal resource usage. Our approach combines wavemap, a hierarchical volumetric map representation, with a novel hierarchical and parallelizable obstacle avoidance algorithm formulated through Riemannian Motion Policies (RMP). Leveraging multi-resolution obstacle avoidance policies, the proposed navigation system facilitates precise, low-latency (36ms), and extremely efficient obstacle avoidance with a very large perceptive radius (30m). We perform extensive statistical evaluations on indoor and outdoor maps, verifying that the proposed system compares favorably to fixed-resolution RMP variants and CHOMP. Finally, the RMP formulation allows the seamless fusion of obstacle avoidance with additional objectives, such as goal-seeking, to obtain a fully-fledged navigation system that is versatile and robust. We deploy the system on a Micro Aerial Vehicle and show how it navigates through an indoor obstacle course. Our complete implementation, called waverider, is made available as open source.

Keywords: Reactive and Sensor-Based Planning , Autonomous Vehicle Navigation , Biologically-Inspired Robots
Abstract: This work explores the use of artificial whiskers as tactile sensors for enhancing the perception and navigation capabilities of mobile robots in challenging settings such as caves and underground mines. These environments exhibit inconsistent lighting conditions, locally self-similar textures, and general poor visibility conditions, that can cause the performance of state-of-the-art vision-based methods to decline. In order to evaluate the efficacy of tactile sensing in this context, three algorithms were developed and tested with simulated and physical experiments: a wall-follower, a navigation algorithm based on Theta*, and a hybrid approach that combines the two. The obtained results highlight the efficacy of tactile sensing for wall-following in intricate environments. When paired with an external method for pose estimation, it further aids in navigating unknown environments. Moreover, by integrating navigation with wall-following, the third, hybrid algorithm enhanced the map traversal speed by roughly 26−43% compared to standard navigation methods without wall-following.

Keywords: Reactive and Sensor-Based Planning , Motion and Path Planning , Autonomous Vehicle Navigation
Abstract: This paper presents a reactive navigation method based on model predictive path integral (MPPI) control with spline interpolation of control input sequence and Stein variational gradient descent (SVGD). MPPI formulates a non-linear optimization problem that determines an optimal control input sequence and solves it using a sampling-based method. Results by MPPI significantly depend on sampling noises. To quickly find paths to avoid large and/or newly detected obstacles, the sampling noises should be set to large. However, the large noises yield non-smooth control input sequences, resulting in non-smooth paths. To prevent this problem, we first introduce spline interpolation of control input sequence in the MPPI process. Owing to the interpolation, smooth control input sequences can be obtained even though the large sampling noises are used. However, a vanilla MPPI algorithm still does not work in a case where there are optimal and near optimal solutions in one scene, e.g., there are several paths to avoid obstacles, because MPPI assumes that a distribution over an optimal control input sequence can be approximated by a Gaussian distribution. To overcome this problem, we further apply SVGD to MPPI with the spline interpolation. SVGD is based on the optimal transportation algorithm and has a property to put samples into around one optimal sample. As a result, we can achieve robust reactive navigation that quickly finds a path to avoid obstacles while keeping smoothness of control input sequence. We validate our proposals in a simulator of a quadrotor. Results show that our proposal involving both the spline interpolation and SVGD outperforms other base line methods.

Keywords: Reactive and Sensor-Based Planning , Manipulation Planning , Motion and Path Planning
Abstract: This paper investigates the problem of effective tool manipulation for motion planning in complex human-like scenarios. Vector-field-based real-time strategies, although widely used, usually do not account for unwieldy tools or incorporate systematic methods to handle these extra maneuvers needed. Instead, we formalize the problem and propose a novel field- based reactive planner that explicitly accounts for rotational forces for seamless maneuvers based on the tool’s geometry and featured points. Furthermore, we capture and encode robot performance through capability metrics and improve the same using an additional quality distribution method. This enables seamless integration of the robot’s embodiment with the reactive force-torque (wrench) field giving rise to flexible tool usage in non-stationary environments. Extensive simulation analysis on a 7 DoF collaborative robot manipulating a common tool in an unorganized table-top layout reinforces our claim of robustness in stationary and non-stationary scenarios.

Keywords: Reactive and Sensor-Based Planning , Motion and Path Planning , Collision Avoidance
Abstract: In this paper, we propose a reactive planning system for quadruped robots based on prescribed-time control. The navigation of the quadruped robot is fundamentally depicted as omnidirectional movements, while a feedback control law is formulated to address any deviations the robot may encounter. In particular, our proposed feedback control system is theoretically proven to achieve convergence within a predefined finite time that is specified by the user. To further compute the optimal convergent time and the local goal state, we present a high-level planning node encompassing terrain-aware kinodynamic search and spatiotemporal trajectory optimization, which can generate collision-free, smooth, and efficient trajectories. The effectiveness of our proposed framework is validated through both numerical simulation and real-robot experiments in indoor and outdoor environments, including scenarios with cluttered obstacles, slopes, and external disturbances.

Keywords: Virtual Reality and Interfaces , Medical Robots and Systems , Surgical Robotics: Planning
Abstract: Robotic-assisted medical systems (RAMS) have gained significant attention for their advantages in alleviating surgeons' fatigue and improving patients' outcomes. These systems comprise a range of human-computer interactions, including medical scene monitoring, anatomical target planning, and robot manipulation. However, despite its versatility and effectiveness, RAMS demands expertise in robotics, leading to a high learning cost for the operator. In this work, we introduce a novel framework using mixed reality technologies to ease the use of RAMS. The proposed framework achieves real-time planning and execution of medical instruments by providing 3D anatomical image overlay, human-robot collision detection, and robot programming interface. These features, integrated with an easy-to-use calibration method for head-mounted display, improve the effectiveness of human-robot interactions. To assess the feasibility of the framework, two medical applications are presented in this work: 1) coil placement during transcranial magnetic stimulation and 2) drill and injector device positioning during femoroplasty. Results from these use cases demonstrate its potential to extend to a wider range of medical scenarios.

Keywords: Learning from Demonstration , Visual Learning , Manipulation Planning , Learning of Geometric Constraints
Abstract: Visual imitation learning provides efficient and intuitive solutions for robotic systems to acquire novel manipulation skills. However, simultaneously learning geometric task constraints and control policies from visual inputs alone Visual imitation learning provides efficient and intuitive solutions for robotic systems to acquire novel manipulation skills. However, simultaneously learning geometric task constraints and control policies from visual inputs alone remains a challenging problem. In this paper, we propose the keypoint-based visual imitation learning (K-VIL) approach that automatically extracts sparse, object-centric, and embodiment-independent task representations from a small number of human demonstration videos. The task representation is composed of keypoint-based geometric constraints on principal manifolds, their associated local frames, and the movement primitives that are then needed for the task execution. Our approach is capable of extracting such task representations from a single demonstration video, and of incrementally updating them when new demonstrations are available. To reproduce manipulation skills using the learned set of prioritized geometric constraints in novel scenes, we introduce a novel keypoint-based admittance controller. We evaluate our approach in several real-world applications, showcasing its ability to deal with cluttered scenes, viewpoint mismatch, new instances of categorical objects, and large object pose and shape variations.

Keywords: Reactive and Sensor-Based Planning , Collision Avoidance , Path Planning for Multiple Mobile Robots or Agents
Abstract: The rise of autonomous driving in everyday life makes efficient and collision-free motion planning more important than ever. However, multi robot applications in highly dynamic environments still pose hard challenges for state-of-the-art motion planners. In this paper, we present a new iteration of a reactive circular fields motion planner with the focus on simultaneous control of multiple robots in robotic soccer games, which is able to operate omnidirectional robots safely and efficiently despite high measurement delays and inaccuracies. Our extension enables the definition and effective execution of complex tasks in soccer specific problems. We extensively evaluated our planner in several complex simulation environments and experimentally verified the approach in realistic scenarios on real soccer robots. Furthermore, we demonstrated the capabilities of our motion planner during the successful participation in the RoboCup 2022 and 2023.

Keywords: Reactive and Sensor-Based Planning , Autonomous Vehicle Navigation
Abstract: We propose a new method for autonomous navigation in uneven terrains by utilizing a sparse Gaussian Process (SGP) based local perception model. The SGP local perception model is trained on local ranging observation (pointcloud) to learn the terrain elevation profile and extract the feasible navigation subgoals around the robot. Subsequently, a cost function, which prioritizes the safety of the robot in terms of keeping the robot's roll and pitch angles bounded within a specified range, is used to select a safety-aware subgoal that leads the robot to its final destination. The algorithm is designed to run in real-time and is intensively evaluated in simulation and real-world experiments. The results compellingly demonstrate that our proposed algorithm consistently navigates uneven terrains with high efficiency and surpasses the performance of other planners. The implementation of our method, including the supplementary video showing the experimental and real-world results, is available at https://rb.gy/3ov2r8.

Keywords: Optimization and Optimal Control , Machine Learning for Robot Control , Industrial Robots
Abstract: The control of granular materials, which are found in many industrial applications, is a challenging open research problem. Granular material systems are complex-behavior (as they could have solid-, fluid-, and gas-like behaviors) and high-dimensional (as they could have many grains/particles with at least 3 DOF in 3D) systems. Recently, a machine learning-based Graph Neural Network (GNN) simulator has been proposed to learn the underlying dynamics. In this paper, we perform optimal control of a rigid body-driven granular material system whose dynamics is learned by a GNN model trained by reduced data generated via a physics-based simulator and Principal Component Analysis (PCA). We use Differential Dynamic Programming (DDP) to obtain optimal control commands that can form granular particles into a target shape. The model and results are shown to be relatively fast and accurate. The control commands are also applied to the ground truth model, i.e., physics-based simulator, to further validate the approach.

Keywords: Optimization and Optimal Control , Machine Learning for Robot Control , Reinforcement Learning
Abstract: This paper presents a novel algorithm for the continuous control of dynamical systems that combines Trajectory Optimization (TO) and Reinforcement Learning (RL) in a single framework. The motivations behind this algorithm are the two main limitations of TO and RL when applied to continuous nonlinear systems to minimize a non-convex cost function. Specifically, TO can get stuck in poor local minima when the search is not initialized close to a “good” minimum. On the other hand, when dealing with continuous state and control spaces, the RL training process may be excessively long and strongly dependent on the exploration strategy. Thus, our algorithm learns a “good” control policy via TO-guided RL policy search that, when used as initial guess provider for TO, makes the trajectory optimization process less prone to converge to poor local optima. Our method is validated on several reaching problems featuring non-convex obstacle avoidance with different dynamical systems, including a car model with 6D state, and a 3-joint planar manipulator. Our results show the great capabilities of CACTO in escaping local minima, while being more computationally efficient than the Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) RL algorithms.

Keywords: Optimization and Optimal Control , Model Learning for Control , Autonomous Vehicle Navigation
Abstract: This work presents a novel Learning Model Predictive Control (LMPC) strategy for autonomous racing at the handling limit that can iteratively explore and learn unknown dynamics in high-speed operational domains. We start from existing LMPC formulations and modify the system dynamics learning method. In particular, our approach uses a nominal, global, nonlinear, physics-based model with a local, linear, data-driven learning of the error dynamics. We conduct experiments in simulation, 1/10th scale hardware, and deployed the proposed LMPC on a full-scale autonomous race car used in the Indy Autonomous Challenge (IAC) with closed loop experiments at the Putnam Park Road Course in Indiana, USA. The results show that the proposed control policy exhibits improved robustness to parameter tuning and data scarcity. Incremental and safety-aware exploration toward the limit of handling and iterative learning of the vehicle dynamics in high-speed domains is observed both in simulations and experiments.

Keywords: Optimization and Optimal Control , Robust/Adaptive Control , Humanoid and Bipedal Locomotion
Abstract: This paper develops a controller synthesis method for ensuring an admissible bound of external forces on biped robots in a desired level. We first introduce the authors` preceding results on the norm-based stability criterion for a biped walking constructed on its linear inverted pendulum model (LIPM). More precisely, an induced norm can be taken to formulate the fact that the balance for a biped robot is achieved if its zero moment point (ZMP) always stays in the supporting region at each step. Based on this norm-based criterion, we aim at making the maximum energy of external forces admissible for balancing the biped robot be a pregiven desired bound gamma(> 0). To achieve this objective, a robust controller is designed through the linear matrix inequality (LMI)-based approach. More importantly, a necessary and sufficient condition for the existence of a robust controller leading to the desired bound is characterized by some LMI conditions. The effectiveness of the overall arguments is validated through some comparative simulation results of a biped walking robot with external forces.

Keywords: Optimization and Optimal Control , Robust/Adaptive Control , Reinforcement Learning
Abstract: This paper investigates the robust optimal control problem of a class of continuous-time, partially linear, interconnected systems. In addition to the dynamic uncertainties resulted from the interconnected dynamic system, unknown bounded disturbances and computational errors are taken into account throughout the learning process, wherein the system’s dynamics are also assumed unknown. These challenges lead the collected online data to be imperfect. In this scenario, traditional data-driven control techniques, such as adaptive dynamic programming (ADP) and robust ADP, encounter a challenge in learning the optimal control policy precisely due to imperfect data. In this paper, a novel data-driven robust policy iteration method is proposed to solve the robust optimal control problems. Without relying on the knowledge of the system’s dynamics, the external disturbances or the complete state, the implementation of the proposed method only needs to access the input and partial state information. Based on the small-gain theorem, the notions of strong unboundedness observability and input-to-output stability, it is guaranteed that the learned robust optimal control gain is stabilizing and that the solution of the closed-loop system is uniformly ultimately bounded despite the existence of dynamic uncertainties and unknown external disturbances. The simulation results reveal the efficiency and practicality of the proposed data-driven control method.

Keywords: Optimization and Optimal Control , Robust/Adaptive Control , Underactuated Robots
Abstract: Optimal behaviours of a system to perform a specific task can be achieved by leveraging the coupling between trajectory optimization, stabilization and design optimization. This approach proves particularly advantageous for underactuated systems, which are systems that have fewer actuators than degrees of freedom and thus require for more elaborate control systems. This paper proposes a novel co-design algorithm, namely Robust Trajectory Control with Design optimization (RTC-D). An inner optimization layer (RTC) simultaneously performs direct transcription (DIRTRAN) to find a nominal trajectory while computing optimal hyperparameters for a stabilizing time-varying linear quadratic regulator (TVLQR). RTC-D augments RTC with a design optimization layer, maximizing the system’s robustness through a time-varying Lyapunov-based region of attraction (ROA) analysis. This analysis provides a formal guarantee of stability for a set of off-nominal states. The proposed algorithm has been tested on two different underactuated systems: the torque-limited simple pendulum and the cart-pole. Extensive simulations of off-nominal initial conditions demonstrate improved robustness, while real-system experiments show increased insensitivity to torque disturbances.

Keywords: Optimization and Optimal Control , Sensorimotor Learning , Whole-Body Motion Planning and Control
Abstract: This paper presents a whole-body robot control method for exploring and probing a given region of interest. The ergodic control formalism behind such an exploration behavior consists of matching the time-averaged statistics of a robot trajectory with the spatial statistics of the target distribution. Most existing ergodic control approaches assume the robots/sensors as individual point agents moving in space. We introduce an approach that decomposes the whole-body of a robotic manipulator into multiple kinematically constrained agents. Then, we generate control actions by calculating a consensus among the agents. To do so, we use an ergodic control formulation called heat equation-driven area coverage (HEDAC) and slow the diffusion using the non-stationary heat equation. Our approach extends HEDAC to applications where robots have multiple sensors on the whole-body (such as tactile skin) and use all sensors to optimally explore the given region. We show that our approach increases the exploration performance in terms of ergodicity and scales well to real-world problems. We compare our method in kinematic simulations with the state-of-the-art and demonstrate the applicability of an online exploration task with a 7-axis Franka Emika robot.

Keywords: Optimization and Optimal Control , Whole-Body Motion Planning and Control
Abstract: We present ReLU-QP, a GPU-accelerated solver for quadratic programs (QPs) that is capable of solving high-dimensional control problems at real-time rates. ReLU-QP is derived by exactly reformulating the Alternating Direction Method of Multipliers (ADMM) algorithm for solving QPs as a deep, weight-tied neural network with rectified linear unit (ReLU) activations. This reformulation enables the deployment of ReLU-QP on GPUs using standard machine-learning toolboxes. We evaluate the performance of ReLU-QP across three model-predictive control (MPC) benchmarks: stabilizing random linear dynamical systems with control limits, balancing an Atlas humanoid robot on a single foot, and performing a whole-body pick-up motion on a quadruped equipped with a six-degree-of-freedom arm. These benchmarks indicate that ReLU-QP is competitive with state-of-the-art CPU-based solvers for small-to-medium-scale problems and offers order-of-magnitude speed improvements for larger-scale problems.

Keywords: Surgical Robotics: Laparoscopy , Computer Vision for Medical Robotics
Abstract: In mini-invasive surgery, the laparoscopic ultrasound probe is visible in the laparoscopic image. We address the problem of estimating the probe pose with respect to the laparoscope without using markers and additional sensors. We propose the first method using a single standard laparoscopic monocular RGB image. It is robust, initialization-free and runs at 10 fps, thus forming a promising tool to improve robotic and augmented reality-based surgery.

Keywords: Surgical Robotics: Laparoscopy , Dual Arm Manipulation , Manipulation Planning , Soft Object Manipulation
Abstract: Robotic manipulation of 3D soft objects remains challenging in both the industrial and medical fields. Various methods based on mechanical modelling, data-driven approaches or explicit feature tracking have been proposed. A unifying disadvantage of these methods is the high computational cost of simultaneous imaging processing, identification of mechanical properties, and motion planning, leading to a need for less computationally intensive methods. We propose a method for autonomous robotic manipulation with 3D surface feedback to solve these issues. First, we produce a deformation model of the manipulated object, which estimates the robots’ movements by monitoring the displacement of surface points surrounding the manipulators. Then we develop a 6-degree-of-freedom velocity controller to manipulate the grasped object to achieve a desired shape. To validate our approach, we conduct comparative simulations with existing methods and perform experiments using phantom and cadaveric soft tissues with the da Vinci Research Kit. The results demonstrate the robustness of the method to occlusions and various materials. Compared to state-of-the-art linear and data-driven methods, our approach is more precise by 46.5% and 15.9%, and saves 55.2% and 25.7% manipulation time, respectively.

Keywords: Surgical Robotics: Laparoscopy , Haptics and Haptic Interfaces , Telerobotics and Teleoperation
Abstract: The da Vinci surgical robot introduced remote control of instruments, providing surgeons with increased dexterity and precision. A major drawback, however, is the loss of sense of touch due to a lack of kinesthetic coupling between the surgical field and the surgeon. This paper presents a framework for sensorless transparency optimized four channel teleoperation. It is sensorless because forces are estimated from existing actuator feedback, with a deep network for dynamics identification. Performance is further optimized by introducing robust acceleration control, with disturbance observers. Experiments performed on the da Vinci Research Kit (dVRK), an open research platform based on the clinically deployed robotic hardware, show improvements in control, force estimation and reflection. The significance is that we demonstrate that high-performance bilateral teleoperation is feasible in clinical systems, without hardware changes, and is available to the dVRK community through a software update.

Keywords: Surgical Robotics: Laparoscopy , Recognition , Kinematics
Abstract: Real-time recognition and prediction of surgical activities are fundamental to advancing safety and autonomy in robot-assisted surgery. This paper presents a multimodal transformer architecture for real-time recognition and prediction of surgical gestures and trajectories based on short segments of kinematic and video data. We conduct an ablation study to evaluate the impact of fusing different input modalities and their representations on gesture recognition and prediction performance. We perform an end-to-end assessment of the proposed architecture using the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) dataset. Our model outperforms the state-of-the-art (SOTA) with 89.5% accuracy for gesture prediction through effective fusion of kinematic features with spatial and contextual video features. It achieves the real-time performance of 1.1-1.3ms for processing a 1-second input window by relying on a computationally efficient model.

Keywords: Surgical Robotics: Laparoscopy , Reinforcement Learning , Autonomous Agents
Abstract: We introduce a novel approach for learning a complex multi-stage needle pick-and-place manipulation task for surgical applications using Reinforcement Learning without expert demonstrations or explicit curriculum. The proposed method is based on a recursive decomposition of the original task into a sequence of sub-tasks with increasing complexity and utilizes an actor-critic algorithm with deterministic policy output. In this work, exploratory bottlenecks have been used by a human expert as convenient boundary points for partitioning complex tasks into simpler subunits. Our method has successfully learnt a policy for the needle pick-and-place task, whereas the state-of-the-art TD3+HER method is unable to achieve success without the help of expert demonstrations. Comparison results show that our method achieves the highest performance with a 91% average success rate.

Keywords: Surgical Robotics: Laparoscopy , Reinforcement Learning , Visual Servoing
Abstract: Surgical tasks such as tissue retraction, tissue exposure, and needle suturing remain challenging in autonomous surgical robotics. One challenge in these tasks is nonprehensile manipulation such as pushing tissue, pressing cloth, and needle threading. In this work, we isolate the problem of nonprehensile manipulation by implementing a vision-based reinforcement learning agent for rolling a block, a task that has complex dynamics interactions, small scale objects, and a narrow field of view. We train agents in simulation with a reward formulation that encourages efficient and safe learning, domain randomization that allows for robust sim2real transfer, and a recurrent memory layer that enables reasoning about randomized dynamics parameters. We successfully transfer our agents from simulation to real and show robust execution of our vision-based policy with a 96.3% success rate. We analyze and discuss the success rate, trajectories, and recovery behaviours for various models that are either using the recurrent memory layer or are trained with a difficult physics environment. Further project information is available at href{https://medcvr.utm.utoronto.ca/ral2023-rollblock.html}{https://medcvr.utm.utoronto.ca/ral2023-rollblock.html}.

Keywords: Surgical Robotics: Laparoscopy , Simulation and Animation , Deep Learning Methods
Abstract: Automation in surgical robotics has the potential to improve patient safety and surgical efficiency, but it is difficult to achieve due to the need for robust perception algorithms. In particular, 6D pose estimation of surgical instruments is critical to enable the automatic execution of surgical maneuvers based on visual feedback. In recent years, supervised deep learning algorithms have shown increasingly better performance at 6D pose estimation tasks; yet, their success depends on the availability of large amounts of annotated data. In household and industrial settings, synthetic data, generated with 3D computer graphics software, has been shown as an alternative to minimize annotation costs of 6D pose datasets. However, this strategy does not translate well to surgical domains as commercial graphics software have limited tools to generate images depicting realistic instrument-tissue interactions. To address these limitations, we propose an improved simulation environment for surgical robotics that enables the automatic generation of large and diverse datasets for 6D pose estimation of surgical instruments. Among the improvements, we developed an automated data generation pipeline and an improved surgical scene. To show the applicability of our system, we generated a dataset of 7.5k images with pose annotations of a surgical needle that was used to evaluate a state-of-the-art pose estimation network. The trained model obtained a mean translational error of 2.59mm on a challenging dataset that presented varying levels of occlusion. These results highlight our pipeline's success in training and evaluating novel vision algorithms for surgical robotics applications.

Keywords: Surgical Robotics: Laparoscopy , Software Architecture for Robotic and Automation , Reinforcement Learning
Abstract: Recent advances in robot-assisted surgery have resulted in progressively more precise, efficient, and minimally invasive procedures, sparking a new era of robotic surgical intervention. This enables doctors, in collaborative interaction with robots, to perform traditional or minimally invasive surgeries with improved outcomes through smaller incisions. Recent efforts are working toward making robotic surgery more autonomous which has the potential to reduce variability of surgical outcomes and reduce complication rates. Deep reinforcement learning methodologies offer scalable solutions for surgical automation, but their effectiveness relies on extensive data acquisition due to the absence of prior knowledge in successfully accomplishing tasks. Due to the intensive nature of simulated data collection, previous works have focused on making existing algorithms more efficient. In this work, we focus on making the simulator more efficient, making training data much more accessible than previously possible. We introduce Surgical Gym, an open-source high performance platform for surgical robot learning where both the physics simulation and reinforcement learning occur directly on the GPU. We demonstrate between 100-5000x faster training times compared with previous surgical learning platforms. The code is available at: https://github.com/SamuelSchmidgall/SurgicalGym.

Keywords: Surgical Robotics: Laparoscopy , Surgical Robotics: Planning
Abstract: Surgical robot task automation has been a promising research topic for improving surgical efficiency and quality. Learning-based methods have been recognized as an interesting paradigm and been increasingly investigated. However, existing approaches encounter difficulties in long-horizon goal-conditioned tasks due to the intricate compositional structure, which requires decision-making for a sequence of sub-steps and understanding of inherent dynamics of goal-reaching tasks. In this paper, we propose a new learning-based framework by leveraging the strong reasoning capability of the GPT-based architecture to automate surgical robotic tasks. The key to our approach is developing a goal-conditioned decision transformer to achieve sequential representations with goal-aware future indicators in order to enhance temporal reasoning. Moreover, considering to exploit a general understanding of dynamics inherent in manipulations, thus making the model's reasoning ability to be task-agnostic, we also design a cross-task pretraining paradigm that uses multiple training objectives associated with data from diverse tasks. We have conducted extensive experiments on 10 tasks using the surgical robot learning simulator SurRoL. The results show that our new approach achieves promising performance and task versatility compared to existing methods. The learned trajectories can be deployed on the da Vinci Research Kit (dVRK) for validating its practicality in real surgical robot settings. Our project website is at: https://med-air.github.io/SurRoL.

Keywords: Safety in HRI , Service Robotics , Software-Hardware Integration for Robot Systems
Abstract: Integrating humanoid service mobile robots into human environments presents numerous challenges, primarily concerning the safety of interactions between robots and humans. To address these safety concerns, we propose a novel approach that leverages the capabilities of digital twin technology by tailoring it to incorporate comprehensive and robust safety concepts. This paper introduces a ``safe-by-design'' digital twin that operates alongside the real twin robot in the loop, engaging real-time safety framework during physical interactions with the surrounding environment, including humans.

To validate the effectiveness of our proposed safe-by-design digital twin framework, we conducted experiments using a humanoid service mobile robot alongside simulated human counterparts. Our results demonstrate the capability of the integrated impact safety module within the proposed digital twin approach to limit the velocities of both the robot's base and arms, adhering to injury biomechanics-based safety thresholds. These findings emphasize the promise of our proposed approach for ensuring the physical safety of humanoid service mobile robots operating in dynamic human environments. It enables the digital twin to preemptively identify potential safety hazards and formulate safe intervention actions to ensure the robot's compliance with safety regulations, paving the way for safer and more widespread adoption of robotic systems in various service domains.

Keywords: Safety in HRI , Learning from Demonstration , Telerobotics and Teleoperation
Abstract: In the field of Learning from Demonstration (LfD), Dynamical Systems (DSs) have gained significant attention due to their ability to generate real-time motions and reach predefined targets. However, the conventional convergence-centric behavior exhibited by DSs may fall short in safety-critical tasks, specifically, those requiring precise replication of demonstrated trajectories or strict adherence to constrained regions even in the presence of perturbations or human intervention. Moreover, existing DS research often assumes demonstrations solely in Euclidean space, overlooking the crucial aspect of orientation in various applications. To alleviate these shortcomings, we present an innovative approach geared toward ensuring the safe execution of learned orientation skills within constrained regions surrounding a reference trajectory. This involves learning a stable DS on SO(3), extracting time-varying conic constraints from the variability observed in expert demonstrations, and bounding the evolution of the DS with Conic Control Barrier Function (CCBF) to fulfill the constraints. We validated our approach through extensive evaluation in simulation and showcased its effectiveness for a cutting skill in the context of assisted teleoperation.

Keywords: Safety in HRI , Human-Aware Motion Planning , Industrial Robots
Abstract: In human-robot collaboration, there has been a trade-off relationship between the speed of collaborative robots and the safety of human workers. In our previous paper, we introduced a time-optimal path tracking algorithm designed to maximize speed while ensuring safety for human workers. This algorithm runs in real-time and provides the safe and fastest control input for every cycle with respect to ISO standards. However, true optimality has not been achieved due to inaccurate distance computation resulting from conservative model simplification. To attain true optimality, we require a method that can compute distances 1. at many robot configurations to examine along a trajectory 2. in real-time for online robot control 3. as precisely as possible for optimal control. In this paper, we propose a batched, fast and precise distance checking method based on precomputed link-local SDFs. Our method can check distances for 500 waypoints along a trajectory within less than 1 millisecond using a GPU at runtime, making it suited for time-critical robotic control. Additionally, a neural approximation has been proposed to accelerate preprocessing by a factor of 2. Finally, we experimentally demonstrate that our method can navigate a 6-DoF robot earlier than a geometric-primitives-based distance checker in a dynamic and collaborative environment.

Keywords: Safety in HRI , Human-Aware Motion Planning , Deep Learning for Visual Perception
Abstract: Handovers play a key role in human-robot interactions. However, current research focuses on visible-hand handovers, thereby heavily relying on hand detection. Large objects in human-robot interactions present a unique challenge: they inherently block the person's hands and arms from the robot's view. This occlusion raises the robot's risk of unintended physical contact with the person, leading to discomfort and safety concerns. This study aims to develop a model that can determine a pose for the robot that ensures a handover that avoids physical contact with the person, especially in scenarios when hands and arms are occluded. Toward this goal, a three-branch multimodal Deep Neural Network (DNN) regression model was implemented. First, a robust human-pose keypoints detection to calculate shoulder-elbow angles is applied. Secondly, we extract the refined object's segmented mask. Thirdly, we compute two intrinsic object properties. The concatenated outputs from these branches pass through extra dense layers, resulting in the prediction of the robot's 14 arms-joint angles. Compared to an only keypoint data processed-based model, our multimodal approach made a 17.7% accuracy improvement. The experiments highlight each pipeline step's significance, showing important results even when hands and arms were heavily occluded, adjusting to different variations.

Keywords: Human-Aware Motion Planning , Safety in HRI
Abstract: Humans have a remarkable ability to fluently engage in joint collision avoidance in crowded navigation tasks despite the complexities and uncertainties inherent in human behavior. Underlying these interactions is a mutual understanding that (i) individuals are prosocial, that is, there is equitable responsibility in avoiding collisions, and (ii) individuals should behave legibly, that is, move in a way that clearly conveys their intent to reduce ambiguity in how they intend to avoid others. Toward building robots that can safely and seamlessly interact with humans, we propose a general robot trajectory planning framework for synthesizing legible and proactive behaviors and demonstrate that our robot planner naturally leads to prosocial interactions. Specifically, we introduce the notion of a markup factor to incentivize legible and proactive behaviors and an inconvenience budget constraint to ensure equitable collision avoidance responsibility. We evaluate our approach against well-established multi-agent planning algorithms and show that using our approach produces safe, fluent, and prosocial interactions. We demonstrate the real-time feasibility of our approach with human-in-the-loop simulations. Project page can be found at https://uw-ctrl.github.io/phri/.

Keywords: Human-Aware Motion Planning , Safety in HRI , Planning under Uncertainty
Abstract: This paper presents a unified prediction and planning algorithm for an autonomous vehicle to interact with an uncertain human-driven vehicle. Predicting human motion is challenging due to inherent uncertainties in diverse human internal states, i.e., driving styles and rationality. To address these complexities, we propose a hierarchical prediction strategy that combines data-driven internal state inference and planning-based human motion prediction. First, we employ Long Short Term Memory Networks (LSTM) based inference modules to capture both driving styles and rationality from the observed motion of human driver. With these inferred internal states, we predict the future trajectories of human-driven vehicle by formulating a human planning model as an optimization problem. Lastly, we present a Stochastic Model Predictive Control (SMPC) for the autonomous vehicle to safely interact with the human-driven vehicle while actively inferring human internal states. The simulation results, demonstrating the lane change scenarios, indicate the proposed method outperforms the existing work in both predicting the human motion and achieving the robot's goal.

Keywords: Safety in HRI , Human-Robot Collaboration , Object Detection, Segmentation and Categorization
Abstract: Perception plays a pivotal role in enhancing the functionality of autonomous agents. However, the intricate relationship between robotic perception metrics and actuation metrics remains unclear, leading to ambiguity in the development and fine-tuning of perception algorithms. In this paper, we introduce a methodology for quantifying this relationship, taking into account factors such as detection rate, detection quality, and latency. Furthermore, we introduce two novel perception metrics for Human-Robot Collaboration safety predicated upon basic perception metrics: Critical Collision Probability (CCP) and Average Collision Probability (ACP). To validate the utility of these metrics in facilitating algorithm development and tuning, we develop an attentive processing strategy that focuses exclusively on key input features. This approach significantly reduces computational time while preserving a similar level of accuracy. Experimental findings demonstrate that integrating this strategy into an object detector results in a notable maximum reduction of 30.09% in inference time and 26.53% in total time per frame. Additionally, the strategy lowers the CCP and ACP in a baseline model by 11.25% and 13.50%, respectively.

Keywords: Safety in HRI , Collision Avoidance , Machine Learning for Robot Control
Abstract: Passivity is necessary for robots to fluidly collaborate and interact with humans physically. Nevertheless, due to the unconstrained nature of passivity-based impedance control laws, the robot is vulnerable to infeasible and unsafe configurations upon physical perturbations. In this paper, we propose a novel control architecture that allows a torque-controlled robot to guarantee safety constraints such as kinematic limits, self-collisions, external collisions and singularities and is passive only when feasible. This is achieved by constraining a dynamical system based impedance control law with a relaxed hierarchical control barrier function quadratic program subject to multiple concurrent, possibly contradicting, constraints. Joint space constraints are formulated from efficient data-driven self- and external C^2 collision boundary functions. We theoretically prove constraint satisfaction and show that the robot is passive when feasible. Our approach is validated in simulation and real robot experiments on a 7DoF Franka Research 3 manipulator.

Keywords: Safety in HRI , AI-Enabled Robotics , Big Data in Robotics and Automation
Abstract: AI-enabled collaboration robots are designed to be used in close collaborations with humans, thus demanding stringent safety standards and quick response times. However, adversarial attacks pose a significant threat to deep learning models of AI-enabled industrial systems, making it crucial to develop methods to improve the models' robustness against them. Adversarial training is one approach to achieve this, but its effectiveness heavily relies on the quality of the training data, which can be expensive to acquire. In this work, we try to balance the need for quality data with the goal to minimize cost

by selecting the most `important' of it for adversarial training. In particular, we propose a task-based robust fast (RAST) learning method that selects training data near to the boundary by considering adversarial samples. Our method improves the speed of model training on

CIFAR-10 by 68.67%, and compared to other data selection methods, has 10% higher accuracy with 10% training data selected, and 7% higher robustness with 4% training data selected. Our method also significantly improves efficiency by at least 25% on adversarial training with the same performance. Finally, we evaluate our method on

a physical robotic arm system with object detection, generating adversarial patches as our attack, and adopting our method as the defense. We find that RAST can defend against 60% of untargeted attacks

and 20% of targeted attacks. Therefore, our work highlights the benefits of

Keywords: Micro/Nano Robots
Abstract: Micro-assembly is an emerging method to fabricate microrobots with multiple modules or particles. However, there is always a lack of a flexible and efficient method to freely create the desired magnetic soft microrobots. In this paper, an automated assembly system based on a two-fingered microhand is presented for fabricating magnetic soft microrobots. Our proposed system can automatically pick and place components to assemble microrobots with a two-fingered micromanipulator, and orient these components through an external magnetic field. The automated assembly has the advantages of high accuracy, high speed, and high success rate. It can endow magnetic microrobots with flexible material selection, arbitrary geometry design, and programable magnetization profile. We can make full use of this system to fabricate multiple magnetic soft microrobots. The experiment results demonstrate that this system can efficiently fabricate microrobots with excellent mechanical properties, which have application potential in robotics, biomedical engineering, and environmental governance.

Keywords: Micro/Nano Robots , Additive Manufacturing
Abstract: In this work, we present a thin-film shape memory alloy (NiTi) microactuator with a magnetic spring. This novel actuator design utilizes two permanent magnets and 3D-printed magnet holders to effectively apply a tensile strain on the NiTi thin-film. This actuator is expected to generate 8.7 mN of blocking force, and a free displacement of 30 μm is experimentally characterized. The actuator leverages bare NiTi film (∼ 1 μm thick) for actuation, enabling a high actuator bandwidth up to 50 Hz. A comprehensive analytical model is also studied, which was then validated by comparing to the experimental results. A launcher mechanism was designed and integrated with the NiTi actuator, and this mechanism was used to launch a microscale projectile (a salt grain) thereby demonstrating the relative high power actuation achievable with thin-film NiTi.

Keywords: Micro/Nano Robots , Automation at Micro-Nano Scales , Biologically-Inspired Robots
Abstract: Wirelessly actuated magnetic microrobots are promising tools in medical applications due to their tiny sizes and attractive robotic properties. However, it remains a huge challenge to integrate sufficient functionalities in a limited volume. Microscopic natural phenomenon is a great reference for current microrobot design, where the underlying intelligence and subtlety spurs related modern artificial systems. Inspired by air bubbles in nature, herein, we report a kind of novel magnetic air bubble microrobots. The air bubble-based structure enables multiple functionalities including cargo delivery, multimode locomotion, micromanipulation, medical imaging, and biosensing. The proposed microrobot is essentially Pickering bubbles composed of magnetic particles and air bubbles. Their hollow structures help produce lighter microrobots with density less than 1 g/cm^3, enabling buoyancy-based self-propulsion. Buoyancy and magnetic forces actuation enables flexible 3D locomotion in fluidic environments. Experimental results show that the microrobots can be controlled properly for designated assignments. Furthermore, the introduction of air bubble enhances ultrasound imaging, facilitating further in vivo applications. These findings offer a significant microrobot design paradigm by exploiting natural physical intelligence at the small scale.

Keywords: Micro/Nano Robots , Automation at Micro-Nano Scales , Mechanism Design
Abstract: In this paper, we introduce magnetic mobile micro-gripping microrobots with two independent actuation modes. By aligning two magnets with slight variations in magnetic moment orientations, we create a net magnetic moment for precise position and orientation control through external fields, while harnessing opposing torques on the magnets to induce internal stresses needed for gripping. Our microrobot design features a compliant spring-like structure for significant deflection, enabling a gripping motion under specific magnetic field conditions. Magnet rotation allows precise control over gripper actions, returning to a default state (normally open or closed) when the magnetic field diminishes. This work advances magnetic field-controlled microrobotics, bridging the millimeter-to-micrometer gap. It holds promise for applications in microsurgery, micro-assembly, and microscale exploration.

Keywords: Micro/Nano Robots , Automation at Micro-Nano Scales , Motion Control
Abstract: Microswimmers have significant potential for medical applications such as long-term drug administration, precise surgery, and so on. The outstanding challenge is to realize power supply, propulsion, and steering mechanisms suitable for operations within the human body and microscale fluids. We

propose the microswimmer composed of a self-propulsive disk-shaped module with multiple channels using biofuel cell (BFC) and electroosmotic propulsion (EOP) and a magnetic rod using magnetic steering (MS). The BFC produces an open-circuit potential (OCP) between

a bioanode and a biocathode by redox reactions. The EOP generates a self-propulsive velocity due to counteracting forces of electroosmotic flows produced by the OCP in the channels arranged between the electrodes. The MS works by aligning the magnetic rod in a controlled magnetic field direction. The prototype was designed and fabricated using an insulating polymer layer, two conductive layers incorporating silver nanoparticles with anodic/cathodic enzymes, and a magnetic layer

containing magnetic nanoparticles. The fast self-propulsion of continuously rotating 30 µm prototypes by the steering in a glucose solution was demonstrated as expected theoretically. This concept has the potential to be used as microrobots for future medical applications

such as a pulling mechanism to assist in guidewire insertion or agents delivering drugs.

Keywords: Micro/Nano Robots , Automation at Micro-Nano Scales , Surgical Robotics: Steerable Catheters/Needles
Abstract: Navigating biomedical instruments inside the brain remains challenging and high-risk. The delicate nature of the tissues involved requires the development of cutting- edge robotic technologies to enhance precision and safety. In response to these demands, this paper presents a novel ribbon-shaped, tendon-driven continuum microrobot designed explicitly to navigate through brain tissues. The microrobot has a cross-sectional area of 1 mm2, and its design is readily compatible with conventional microfabrication techniques for further miniaturization. The flat geometry aims to provide superior maneuverability and opens up new challenges for modeling and control. We detail the design methodology and fabrication, followed by in vitro characterization and testing within brain tissue phantoms.

Keywords: Micro/Nano Robots , Biologically-Inspired Robots , Additive Manufacturing
Abstract: Selective control mechanisms of microrobots have attracted significant attention from researchers. So far, selective control within multiple/swarm magnetic microrobots has been achieved with many strategies, such as utilizing locally specified magnetic fields, applying electrostatic anchoring, taking the advantages of geometry/wettability heterogeneity of the microrobots, etc. Using the step-out behavior of helical microrobots driven by a rotating magnetic field, researchers have proposed a mathematical model for multihelical motor that can be selectively controlled. Based on this model, we developed a micromotor that consists of three geometrically heterogeneous helices that can be selectively driven within a specific narrow frequency range. This type of micromotor shows bi-direction motion capability and has the potential to be used as an actuation unit for multiple types of functional micromechanisms.

Keywords: Space Robotics and Automation , Calibration and Identification
Abstract: Space manipulator system (SMS) maneuvers can excite flexible appendages, while fuel sloshing effects impact its dynamics and performance. To predict this behavior and control such systems, sloshing and flexible appendages are modeled. A novel system identification scheme is developed, which identifies all parameters required for the reconstruction of system dynamics despite unmeasurable sloshing and modal states. This is achieved by two identification experiments. In Exp.1 all unmeasurable states are eliminated, while in Exp.2 the unmeasurable sloshing states are eliminated, and a novel estimator is used for the unmeasurable modal states. The significance of accurate SYSID in controller design and performance is demonstrated by simulating a 3D SMS controlled by model-based and robust controllers. In both cases, using the identified parameters results in significant robust control performance enhancement.

Keywords: Space Robotics and Automation , Compliance and Impedance Control , Mechanism Design
Abstract: The construction of large-scale space facilities requires the use of on-orbit construction technology. However, several of its key components, such as standard interface design, compliant control methods, and path planning for multi-branch robots, still need improvement before practical application. This paper presents a comprehensive solution for on-orbit construction tasks, encompassing a novel standard interface, docking control method, and path planning method for space multi-branch robots. Firstly, a novel standard interface is introduced, which features multiple mating modes and a lightweight design. Additionally, a compliant docking method is provided to generate lower contact forces along the Z-direction. Furthermore, for four-armed space robots, a hierarchical planning method is proposed, which innovates in environment map construction and locomotion planning. Specifically, the closedform Minkowski sum method is employed to solve the robot’s free space, and a concise locomotion method is elucidated based on transition support points. Finally, simulations and experiments are conducted.

Keywords: Space Robotics and Automation , Compliant Joints and Mechanisms , Mechanism Design
Abstract: The exploration of the lunar poles and the collection of samples from the martian surface are characterized by shorter time windows demanding increased autonomy and speeds. Autonomous mobile robots must intrinsically cope with a wider range of disturbances. Faster off-road navigation has been explored for terrestrial applications but the combined effects of increased speeds and reduced gravity fields are yet to be fully studied. In this paper, we design and demonstrate a novel fully passive suspension design for wheeled planetary robots, which couples for the first time a high-range passive rocker with elastic in-wheel coil-over shock absorbers. The design was initially conceived and verified in a reduced-gravity (1.625 m/s ² ) simulated environment, where three different passive suspension configurations were evaluated against steep slopes and unexpected obstacles, and later prototyped and validated in a series of field tests. The proposed mechanically-hybrid suspension proves to mitigate more effectively the negative effects (high-frequency/high-amplitude vibrations and impact loads) of faster locomotion (~1 m/s) over unstructured terrains under varied gravity fields.

Keywords: Space Robotics and Automation , Compliant Joints and Mechanisms , Soft Robot Materials and Design
Abstract: Spherical robots are a different type of mobility platform. A spherical robot is self-contained within its shell rather than relying on a chassis with wheels to navigate. In this shell, it is completely shielded from dust and the environment. This benefit of geometric simplicity has led to the spherical robot becoming an advantageous option for all-terrain exploration and surveying. This paper focuses on a novel iteration of such a robot with a pressurized pneumatic shell design. A soft robot of this type brings benefits of a passive, compliant contact surface that can affect its performance. However, the added softness of its shell adds new unmodeled dynamics into the system that impair commonly used control schemes. This paper outlines the design and manufacture of a soft, inflatable, spherical shell designed for a robot driven by an internal 2-DOF pendulum. In addition, it presents models for controlling the pendulum and understanding the shell dynamics. The paper concludes with experimental validations of these models and field tests of the system on slopes, gravel, rough grass, and on water.

Keywords: Space Robotics and Automation , Mechanism Design , Engineering for Robotic Systems
Abstract: Exploration missions on extra terrestrial celestial bodies are to date performed by complex and heavy robotic systems. The trend is towards lighter modular systems that can be (re)configured in situ according to mission specific requirements. To facilitate flexible configurability, a multifunctional interconnect is used to mechanically couple the involved systems while providing electrical power and data transmission. The paper presents the further development of the reliable electro-mechanical interface (EMI) from the TransTerrA project, which has been proven in several field tests and reached TRL 4. Docking under loads of up to 550 N has been successfully tested with the new design. The experiments presented include undocking at various inclinations with different loads expected for the application scenario.The maximum determined static load that can be carried by the further developed EMI is 2000 N. In further experiments, new contact blocks responsible for the transfer of electrical power and data were tested for water resistance and resilience to environmental factors, as well as power and data transfer.The obtained results will be helpful in the development of a multi-functional interface suitable for lunar applications and missions having similar challenging environmental conditions